Tech News - Data

Yesterday, the team at Redis Labs, the provider of Redis Enterprise announced that its customers can now scale their datasets using Intel Optane DC persistent memory. Scaling will be offered cost-effectively at a  multi-petabyte scale, at sub-millisecond speeds. Also, the two-day RedisConf2019 (2-3 April) was held at San Francisco, yesterday, where 1500 Redis developers, innovators and contributors shared their use cases and experiences. Redis Enterprise, a linearly scalable, in-memory multi-model database, supports native and probabilistic data structures, AI, streams, document, graph, time series, and search. It has been designed and optimized to be operated in either mode of Intel’s persistent memory technology that is, Memory Mode and App Direct Mode. Redis Enterprise offers the customers flexibility for using the most effective mode to process their massive data sets quickly and cost-effectively. Intel Optane DC persistent memory is a memory technology that provides a combination of affordable large capacity and support for data persistence. Redis Labs collaborated closely with Intel throughout the development of Intel Optane DC persistent memory for providing high-performance to the Redis Enterprise database. Also, it drastically improved performance in Benchmark testing while offering huge cost savings at the same time. The benchmark testing conducted by various companies to test Intel Optane DC persistent memory, reveals that Redis Enterprise has proved that a single cluster node with a multi-terabyte dataset can support over one million operations per second at sub-millisecond latency while also serving over 80% of the requests from persistent memory. Redis Enterprise on Intel Optane DC persistent memory also offered more than 40 percent cost savings as compared to the traditional DRAM-only memory. Key features of Intel Optane DC persistent memory It optimizes in-memory databases for advanced analytics in multi-cloud environments. It reduces the wait time associated with fetching the data sets from the system storage. It also helps in transforming the content delivery networks while bringing in greater memory capacity for delivering immersive content at the intelligent edge and provides better user experiences. It provides consistent QoS (Quality of Service) levels in order to reach out to more customers while managing TCO (Total Cost of Ownership) both from hardware and operating cost levels. It also provides cost-effective solutions for customers. Intel Optane DC persistent memory provides with a persistent memory tier between DRAM and SSD that provides up to 6TBs of non-volatile memory capacity in a two-socket server and up to 1.5TB of DRAM. Moreover, it extends a standard machine’s memory capacity to 7.5TBs of byte-addressable memory (DRAM + persistent memory), while also providing persistence. This technology is available in a DIMM form factor and as a 128, 256, and 512GB persistent memory module. Alvin Richards, chief product officer at Redis Labs wrote to us in an email, “Enterprises are faced with increasingly massive datasets that require instantaneous processing across multiple data-models. With Intel Optane DC persistent memory, combining with the rich data models supported by Redis Enterprise, global enterprises can now achieve sub-millisecond latency while processing millions of operations per second with affordable server infrastructure costs.” He further added, “Through our close collaboration with Intel, Redis Enterprise on Intel Optane DC persistent memory our customers will not have to compromise on performance, scale, and budget for their multi-terabyte datasets.” Redis Enterprise is available for any cloud service or as downloadable software for hardware along with Intel Optane DC persistent memory support. To know more about Intel Optane DC persistent memory, check out the Intel’s page. Announcements at RedisConf 19 Yesterday at the RedisConf19, Redis Labs introduced two new data models and a data programmability paradigm for multi-model operation. The company made major announcements including Redis TimeSeries, RedisAI and RedisGears. RedisTimeSeries Redis TimeSeries is designed to collect and store high volume and velocity data and organize it by time intervals. It helps organizations to easily process useful data points with built-in capabilities for downsampling, aggregation, and compression. This provides organizations with the ability to query and extract data in real-time for analytics. RedisAI RedisAI eliminates the need to migrate data to and from different environments and it allows developers to apply state-of-the-art AI models to the data. RedisAI reduces processing overhead by integrating with common deep learning frameworks including TensorFlow, PyTorch, and TorchScript, and by utilizing Redis Cluster capabilities over GPU-based servers. RedisGears RedisGears, an in-database serverless engine, can operate multiple models simultaneously. It is based on the efficient Redis Cluster distributed architecture and enables infinite programmability options supporting event-driven or transaction-based operations. Today, Redis Labs will be showing how to get the most of Redis Enterprise on Intel’s persistent memory at RedisConf19. Redis Labs moves from Apache2 modified with Commons Clause to Redis Source Available License (RSAL) Redis Labs announces its annual Growth of more than 60% in the Fiscal Year 2019 Redis Labs raises $60 Million in Series E Funding led by Francisco partners

Databricks announces Databricks Runtime 4.2 with numerous updates and added components on Spark internals, Databricks Delta and improvisions to its previous version. The databricks runtime 4.2 is powered with Apache Spark 2.3 and recommended for its quick adoption to enjoy the upcoming GA release of Databricks Delta. Databricks Runtime is a set of software artifacts which runs on the clusters of machines and improves the usability and performance of big data analytics. New Features of Databricks Runtime 4.2 Added Multi-cluster writing support, enabling users to use the transactional writing features from Databricks Delta. Streams getting recorded directly to the registered table on Databricks Delta. These streams are stored in the Hive metastore of Databricks Delta platform using df.writeStream.table(...). Added new streaming foreachBatch() for Scala. This helps to define a function for processing output of every micro batch using DataFrame operations. Added support for streaming foreach() for Python language which was earlier available only to Scala. Added from_avro/to_avro functions to support read/write Avro data within DataFrame. Improvements All commands and queries of Databricks Delta support referring to a table using its path as an identifier (that is, delta.`/path/to/table`). DESCRIBE HISTORY includes commit ID and is now ordered newest to oldest by default. Bug Fixes Partition-based filtering predicates operate correctly for special cases like when the predicates differ from the table. Fixed missing column AnalysisException for performing better equality checks on boolean columns in Databricks Delta tables i.e. booleanValue = true. Stopped modifying transaction log while using CREATE TABLE for creating a pointer to an existing table. This prevents unnecessary conflicts with concurrent streams and allows the creation of metastore pointer to tables where the user only has read access to the data. Stopped causing Out Of Memory in the driver while Calling display() on a stream with large amounts of data. Fixed truncation of long lineages which were earlier causing StackOverFlowError while updating the state of a Databricks Delta table. For more details, please read the release notes officially documented by Databricks. Databricks open sources MLflow, simplifying end-to-end Machine Learning Lifecycle Project Hydrogen: Making Apache Spark play nice with other distributed machine learning frameworks Apache Spark 2.3 now has native Kubernetes support!

DeepBrain Chain project, AI bot that challenges humans, usql v0.6.0, and more in today’s top stories around machine learning, deep learning,and data science news. 1. DeepBrain Chain, the First AI Computing Platform Driven by Blockchain DeepBrain Chain, is the first AI computing platform driven by blockchain for global AI computing resource sharing and scheduling. As most organizations do not have capital to buy expensive GPU servers,  these companies provide a huge number of GPU servers which are unused or idle for a prolonged period.   The DeepBrain Chain provides a decentralized AI Computing platform, which is low cost, private, flexible, and safe. It serves the interests of several parties such as, the Miner’s main income is rewarded with token from mining, the AI companies just pay small amounts to run. Also, the Chain uses the smart contract in order to physically separate the data provider and data trainer. Thus, it protects the data of the provider. The interests of three major parties can be reconciled with the advanced technology. It is also automatically adjustable; if some nodes of DBC are attacked by hackers, the remaining nodes are working well as usual. DBC makes sure AI factories’ operations will never be interrupted. 2. Alibaba develops an AI bot to challenge humans in comprehension Alibaba, China’s biggest online e-commerce, has developed a deep neural network model, which has out-performed humans in a global reading comprehension test. According to a release, the model has scored higher on the Stanford Question Answering dataset (a large-scale reading comprehension test with more than 10,000 questions). Alibaba’s machine-learning models scored 82.44 on the test, compared with 82.304 by humans, on 11th January.  Si Luo, chief scientist of natural language processing at Alibaba’s research arm, said that, “We believe the underlying technology can be gradually applied to numerous applications such as customer service, museum tutorials, and online response to inquiries from patients, freeing up human efforts in an unprecedented way”. Similar to the model’s performance in the Stanford test, the machine learning model could identify the questions raised by consumers and look for the most relevant answers from prepared documents. Currently, the system only works best with questions that offer clear-cut answers. If the language or expressions are too vague, has grammatical errors, or there is no prepared answer, the bot may not work properly. 3. usql v0.6.0 released A universal command-line interface for SQL databases releases its version 0.6.0 with major updates below. Syntax highlighting Better compatibility with psql commands Homebrew support The release also includes some minor feature additions and a general code cleanup. Know more about this release on GitHub. 4. Cumulative Update #3 for SQL Server 2017 RTM Microsoft has released the 3rd cumulative update for SQL Server 2017 RTM. The major changes include: CPU timeout setting added to Resource Governor workgroup Support for MAXDOP option added for CREATE STATISTICS and UPDATE STATISTICS statements in SQL Server 2017 Improvement in tempdb spill diagnostics in DMV and Extended Events in SQL Server 2017 XML Showplans can now provide a list of statistics used during query optimization in SQL Server 2017 PolyBase technology enabled in SQL Server 2017 Execution statistics of a scalar-valued, user-defined function added to the Showplan XML file in SQL Server 2017 Optimizer row goal information added in query execution plans in SQL Server 2017 Other fixes and updates can be found here. The update can be downloaded from the Microsoft Download Center. Registration is no longer required to download the Cumulative updates. 5. Logz.io: AI-Powered ELK as a Service Logz.io have launched an AI powered ELK as a cloud service solution which offers a fully managed environment and unlimited data with automatic data parsing capabilities. The ELK stack (for Elasticsearch, Logstash, and Kibana — now called the Elastic Stack) is basically used to handle operational data, specifically log files. Although open source, this stack can be hard to implement and manage according to enterprise standard and is often expensive due to its labor-intensive nature. Logz.io’s ELK as a cloud service solution is an enhanced architecture which delivers the advanced log analytics, integration and security that enterprises require. The platform has an ‘intelligence layer’ which applies artificial intelligence to optimize data, establish correlations between new deployments and resulting log errors, and identify undetected patterns in the data, among other uses.

Peer-to-peer file services like BitTorrent were seeing a decline in traffic over the last decade, with the increasing popularity of affordable streaming services like Netflix. The number of piracy streaming options fell and was said to be accounted for the services provided by the on-demand video services and strict anti-piracy laws. However, recent reports published by Sandvine, who has been keeping a close eye on the file-sharing traffic, suggested that the traffic for streaming file-services is now growing. The report also noted that BitTorrent was earlier evidently losing out on market share but is now emerging to be the leading file-sharing place with 97% share. Over the past few years, Netflix had emerged as the leader in the streaming market and was single-handedly responsible for hosting a wide variety of content. While this was happening, the market observed a decrease in upstream and downstream file sharing of content, popularly called the torrents. This went on to suggest that the streaming market was rising and piracy downloading was reducing. However, Netflix is no more the single most popular streaming services that exist. Other video streaming services like Amazon Prime videos, Hulu, and others have fragmented the market with popular content playing over multiple services. This seems to have re-introduced the traditional practice of file-sharing over the internet and BitTorrent has gained the market. Sandvine's report further stated the following in its report: “More sources than ever are producing ‘exclusive’ content available on a single streaming or broadcast service – think Game of Thrones for HBO, House of Cards for Netflix, The Handmaid’s Tale for Hulu, or Jack Ryan for Amazon. To get access to all of these services, it gets very expensive for a consumer, so they subscribe to one or two and pirate the rest.” File sharing involves uploading and downloading a part of a file over a peer-to-peer network. Reportedly, file sharing makes up for 3% of all download and 22% of all upload traffic. Here, BitTorrent enjoys the lion's share with a massive 97% of all upstream file-sharing traffic. Evidently, fragmentation in the subscription video-on-demand market is playing the major role. To know more about this in detail, check out the discussion thread on Hacker News. The cryptocurrency-based firm, Tron acquires BitTorrent Facebook Watch is now available world-wide challenging video streaming rivals, YouTube, Twitch, and more Implementing fault-tolerance in Spark Streaming data processing applications with Apache Kafka

Last week, the Internet Corporation for Assigned Names and Numbers (ICANN) decided to call for the full deployment of the Domain Name System Security Extensions (DNSSEC) across all unsecured domain names. ICANN took this decision because of the increasing reports of malicious activity targeting the DNS infrastructure. According to ICANN, there is an ongoing and significant risk to key parts of the Domain Name System (DNS) infrastructure. The DNS that converts numerical internet addresses to domain names, has been the victim of various attacks by the use of different methodologies. https://twitter.com/ICANN/status/1099070857119391745?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Etweet Last month security company FireEye revealed that hackers associated with Iran were hijacking DNS records, by rerouting users from a legitimate web address to a malicious server in order to steal passwords. This “DNSpionage” campaign, was targeting governments in the United Arab Emirates and Lebanon. The Homeland Security’s Cybersecurity Infrastructure Security Agency had warned that U.S. agencies were also under attack. In its first emergency order amid a government shutdown, the agency ordered federal agencies to take action against DNS tampering. David Conrad, ICANN’s chief technology officer told the AFP news agency that the hackers are “going after the Internet infrastructure itself.” ICANN is urging domain owners for deploying DNSSEC, which is a more secure version of DNS and is difficult to manipulate. DNSSEC cryptographically signs data which makes it more difficult to be spoofed. Some of the attacks target the DNS where the addresses of intended servers are changed with addresses of machines controlled by the attackers. This type of attack that targets the DNS only works when DNSSEC is not in use. ICANN also reaffirms its commitment towards engaging in collaborative efforts for ensuring the security, stability, and resiliency of the internet’s global identifier systems. This month, ICANN offered a checklist of recommended security precautions for members of the domain name industry, registries, registrars, resellers, and related others, to proactively take steps to protect their systems. ICANN aims to assure that internet users reach their desired online destination by preventing “man in the middle” attacks where a user is unknowingly re-directed to a potentially malicious site. Few users have previously been a victim of DNS hijacking and think that this move won’t help them out. One user commented on HackerNews, “This is nonsense, and possibly crossing the border from ignorant nonsense to malicious nonsense.” Another user said, “There is in fact very little evidence that we "need" the authentication provided by DNSSEC.” Few others think that this might work as a good solution. A comment reads, “DNSSEC is quite famously a solution in search of a problem.” To know more about this news, check out ICANN’s official post. Internet governance project (IGP) survey on IPV6 adoption, initial reports Root Zone KSK (Key Sign Key) Rollover to resolve DNS queries was successfully completed RedHat shares what to expect from next week’s first-ever DNSSEC root key rollover

Hortonworks proudly announces the eagerly awaited full release of its data platform,  Hortonworks Data Platform 3.0. With businesses becoming more data-driven, Hortonworks Data Platform 3.0 (HDP 3.0) is a major footstep in the plan for dominating the Big Data ecosystem. They’ve made major changes within its stack and expanded their ecosystem to include trending technologies like Deep Learning. With the GA release, HDP 3.0 equips businesses with enterprise-grade functionalities, enabling speedy application deployment, managing machine learning workloads with real-time database management. The platform is designed to provide complete security and governance for your business applications. The data platform is added with additional support to GPU computing, containerization, Namenode Federation and Erasure Coding and all these new features are developed on Hadoop 3.1. The platform supports both on-premise and cloud deployment including major cloud platforms such as Amazon Web Services, Microsoft Azure, and Google Cloud. The platform is also stocked with Apache Ranger and Apache Atlas to provide a secure and trusted Data Lake infrastructure. To keep the stack intact and smooth, the new release has deprecated various Apache components like Falcon, Mahout, Flume and Apache Hue. Key features of HDP 3.0: Agile Application Deployment: It enables application developers to deploy their applications using containerization technology. With this, developers can test new versions and at the same time can create new features without damaging the old ones. This feature results in speedy application deployment along with optimum utilization of resources at hand. Deep Learning Support: With Deep Learning technology becoming the backbone of today’s intelligence, HDP 3.0 provides complete support for GPU computing and deep learning workloads. The platform provides both GPU pooling and GPU isolation support through which GPU resources can be used at the optimal level and at the same time can be used exclusively for a specific application based on its priority and complexity level. Cloud Optimization: It is accelerated with automated cloud provisioning for simpler deployment of big data applications with support to major cloud object stores such as Amazon S3, Azure Data Lake, and Google Cloud Storage. The platform also provides speedy query performance with the support of cloud connectors including Apache HBase and Apache Spark. This newly revamped and innovated big data platform can help businesses achieve faster insights and with decision-making in today's competitive business environment. For more detailed information on the HDP 3.0, please visit the official product page. Hortonworks partner with Google Cloud to enhance their Big Data strategy Why AWS is the preferred cloud platform for developers working with big data? Getting to know different Big data Characteristics

Ethereum 2.0 is coming really soon and it could increase the Ethereum network’s capacity to process transactions by a 1000 times. In the annual Ethereum developer conference- Devcon, Vitalik Buterin, the creator of this second largest blockchain announced that the update which was formerly known as Ethereum 2.0 is now called ‘Serenity’. Buterin also addressed the massive efforts that have been put to upgrade the network in the past, especially with issues like the DAO hack and  “super-quadratic sharding” that bogged the team down. What can we expect in Serenity? “We have been actively researching, building, and now, finally getting them all together” -Vitalik Buterin In the month of September, Danner Langley, senior blockchain developer at Rocket Pool revealed the roadmap for Ethereum 2.0. ‘Serenity’ will encompass multiple projects that Ethereum developers have been working on since 2014. It will see Ethereum finally switch from ‘proof-of-work’ to ‘proof-of-stake’. This is a model in which people and organizations holding ether will “stake” their own coins in order to maintain the network. They will earn block rewards for doing so. This will also help to achieve a sharded blockchain verifying data on the network, thus increasing overall efficiency. The new upgrade will also make the network much faster, more secure, less energy-intensive and capable of handling thousands of transactions per second. Serenity will include eWASM, which is a replacement to the existing Ethereum Virtual Machine (EVM) used to compile the smart-contracts. eWASM will double the transaction throughput rate as compared to EVM. He also added that before the official launch of Serenity, developers will make some final tweaks including stabilizing protocol specifications and cross-client testnets. Buterin believes Ethereum will soar with the Serenity upgrade. During the conference, Buterin said that Serenity will be introduced in 4 phases: Phase one will include an initial version with proof-of-stake beacon chain. This would co-exist alongside  Ethereum itself and will allow Casper validators to participate. Phase two will represent simplified version of Serenity with limited features. Excluding smart contracts or money transfers from one shard to another. Phase three will be an amplified version of Serenity with cross-shard communication where users can send funds and messages across different shards. Phase four will have the final tweaks and optimized features Is Vitalik Buterin taking a backseat? In a conversation with MIT Technology Review, Buterin said that  it’s time for him to start fading into the background as “a necessary part of the growth of the community.” Taking the cue from Ethereum being decentralized, where a single component failure could not bring down the whole system, Buterin is  “out of the decision-making in a lot of ways,” said Hudson Jameson of the Ethereum Foundation. This will pave the way for the community to thrive and become more decentralized. Buterin says that his involvement in the project has amounted to “a significantly smaller share of the work than I had two or three years ago,” also adding that downsizing his influence is “something we are definitely making a lot of progress on.” Ethereum’s development will not end with Serenity, since important issues such as transaction fees and governance are still yet to be addressed. Buterin and his team have already begun planning future tweaks along with more tech improvements. To know more about this news, head over to OracleTimes. Aragon 0.6 released on Mainnet allowing Aragon organizations to run on Ethereum Mainnet Vitalik Buterin’s new consensus algorithm to make Ethereum 99% fault tolerant  

Machine Learning has energised software applications with highly accurate predictions thereby upsurging the product demand of tech driven companies. However, while developing such smart applications, numerous machine learning challenges and software development issues are been faced by data scientist and machine learning professionals. Today, Databricks open sources their newly developed framework MLflow, with an aim to simplify their complex machine learning experiments with smart automation and numerous accessibility in deploying your machine learning models across any platform. With MLflow, Machine Learning users can simply standardize their complex processes while building and deploying their machine learning and predictive models. With this framework, data scientists are fueled with lots of automation accessibility through which they can track experiments, package their machine learning codes and manage their models on any of the popular machine learning frameworks. The current platform offers following three components: MLflow Tracking: This component allows you to log codes, data files, config and results. It also allows to query your experiments through which you visualize and compare your experiments and parameters swiftly without much hassle. MLflow Projects: It provides structured format for packaging machine learning codes along with useful API and CLI tools.This allows data scientists to reuse and reproduce their codes and easily chain their projects and workflows together. MLflow Models: It is a standard format for packaging and distributing machine learning models across different downstream tools. Azure ML compatible models, Deploying with Amazon Sagemaker or deploying on a local REST API are some of the examples of distributing models. The current version is just an Alpha release and more features would be added to its full release. To get more details on its core offerings, APIs and command-line interfaces, read the official documentation at mlflow.org. MachineLabs, the browser based machine learning platform, goes open source Microsoft Open Sources ML.NET, a cross-platform machine learning framework Google announces Cloud TPUs on the Cloud Machine Learning Engine (ML Engine)

The Apache Ignite community has announced the latest version of Apache Ignite, its open-source distributed database. Apache Ignite 2.4 features new machine learning capabilities, Spark DataFrames support, and the introduction of a low-level binary client protocol. Machine Learning APIs were first teased at the launch of Apache Ignite 2.0, approximately eight months ago. Now with Apache Ignite 2.4, the ML Grid is production ready. With new ML features, Ignite users can deal with fraud detection, predictive analytics, and for building recommendation systems. The ML grid can also solve regression and classification tasks,  and avoid ETL from Ignite to other systems. ML Grid in the future releases of Ignite 2.4, will also incorporate a genetic algorithm software, donated by NetMillennium Inc. This software will help in solving optimization problems by simulating the process of biological evolution. These in turn can be applied to real-world applications including automotive design, computer gaming, robotics, investments, traffic/shipment routing and more. There is also a good news for Spark users. Dataframes is now officially supported for Apache Spark. In addition, Apache Ignite can also be installed from the official RPM repository. Apache Ignite 2.4 also has a new low-level binary client protocol. This would allow all developers, including but not limited to Java, C#, and C++ developers, to utilize Ignite APIs in their applications. The protocol communicates with an existing Ignite cluster without starting a full-fledged Ignite node. An application can connect to the cluster through a raw TCP socket from any programming language. Apache Ignite 2.4 took five months in total for development. Normally, a new version is rolled out every three months. You can read the complete list of addition in the release notes.

AT&T, T-Mobile, and Sprint sold their customers’ real-time location data to a bounty hunter, as reported by Motherboard in January. As per the reports, Motherboard was even able to purchase the real-time location of a T-Mobile phone from a bounty hunter source on the black market for $300. Telecom companies responded that this abuse was a fringe case. However, in reality, around 250 bounty hunters and related businesses had access to AT&T, T-Mobile, and Sprint customer location data. As per the documents by CerCareOne, a location data seller that operated until 2017, one of the bail bond firms was using the phone location service more than 18,000 times, and others were using it thousands or tens of thousands of times. These documents include the list of companies that had access to the data and also the phone numbers that were pinged by those companies. According to the documents, the location requests stretch from 2012 up to 2017, with some of the phones being located multiple times over minutes, hours, and days. CerCareOne sold cell phone tower data and also highly sensitive and accurate GPS data to bounty hunters. This data was so precise that users could easily locate someone’s location inside a building. CerCareOne operated in secrecy for almost 5 years by making its customers agree to “keep the existence of CerCareOne.com confidential,” according to terms of use document obtained by Motherboard. The company allowed bounty hunters, bail bondsmen, and bail agents to find the real-time location of mobile phones and it would sometimes charge up to $1,100 per phone location. Oregon Senator Ron Wyden said in an emailed statement after presented with Motherboard’s findings, “This scandal keeps getting worse. Carriers assured customers location tracking abuses were isolated incidents. Now it appears that hundreds of people could track our phones, and they were doing it for years before anyone at the wireless companies took action. That’s more than an oversight hat’s flagrant, willful disregard for the safety and security of Americans.” In an email to Motherboard, Eva Galperin, director of cybersecurity at campaign group the Electronic Frontier Foundation said, “The scale of this abuse is outrageous.” The target phones received no text message warning that they were being tracked. Previously telecom companies and location aggregators have told Motherboard that they require clients to obtain consent from people they wish to track. A Sprint spokesperson wrote in an email, “We contractually require location aggregators to obtain prior written consent from Sprint 60 days before the use of any sub-aggregator, and we received no such request related to CerCareOne,” 15 senators called on the FCC and Federal Trade Commission for investigating as to how consumers location data ended up in the hands of bounty hunters. An FCC spokesperson told Motherboard in an email, “We are investigating carriers’ handling of location information, and we can’t comment on what facts we have uncovered in the middle of an active investigation.” Senator Mark Warner, presented with Motherboard’s new findings, said in a statement that “we have a systemic problem across the digital economy, where consumers remain totally in the dark about how their data is collected, sold or shared, and commercialized.” To know more, check out Motherboard’s post. Internal memo reveals NASA suffered a data breach compromising employees social security numbers U.S. Senator introduces a bill that levies jail time and hefty fines for companies violating data breaches Former Senior VP’s take on the Mariott data breach; NYT reports suspects Chinese hacking ties

Yesterday, the team at Aragon, announced the release of Aragon 0.6, named Alba, on Ethereum Mainnet. It’s now possible to create Aragon organizations on the Ethereum Mainnet. Earlier, the organizations were running on Ethereum testnets, without real-world inferences. Aragon 0.5 was released seven months ago and since then, more than 2,500 organizations have been created with it. The total number of Aragon organizations have now crossed 15,000. Aragon 0.5 was the first release to get powered by AragonOS. This release was only deployed on the Rinkeby Ethereum Testnet. Major updates in Aragon 0.6 Permissions Permissions are a dynamic and powerful way to customize the organization. They manage who can access resources on your organization, and how. For example, one can create an organization in which, funds can be withdrawn only after the voting is done. The votes can be only created by a board of experts, allowing anyone in the organization to cast votes. Peers can also vote to create tokens to add new members. Possibilities are endless with ‘Permissions’ as any governance process could be now implemented. Source: Aragon 2. Voting gets easier Voting enables participation and collaborative decision-making. The team at Aragon have rebuilt the card-based voting interface from the ground up. This interface helps one to a look at the votes at a glance. Source: Aragon 3.  AragonOS 4 Aragon 0.6 features AragonOS 4, a smart contract framework for building DAOs, dapps and protocols. The AragonOS 4 is yet to be released but has managed to create some buzz. Its architecture is based on the idea of a decentralized organization as an aggregate of multiple applications. The architecture also involves the use of the Kernel which governs how these applications can talk to each other and also how other entities can interact with them. AragonOS 4 makes the interaction with Aragon organizations even more secure and stable. It’s easy to create your own decentralized organization now. You can start by choosing the network for your organization and follow the next steps on Aragon’s official website. Note: The official blog post suggests to not place large amounts of funds in Aragon 0.6 organizations at this point as there might be some unforeseen situations where user funds could be at risk. Read more about Aragon 0.6 on the Aragon’s official blog post. Stable version of OpenZeppelin 2.0, a framework for smart blockchain contracts, released! IBM launches blockchain-backed Food Trust network which aims to provide greater transparency on food supply chains 9 recommended blockchain online courses

On Monday, at the Web Summit 2018, Darren Aronofsky in his interview with WIRED correspondent Lauren Goode, spoke about how virtual reality and artificial intelligence is giving filmmakers and writers the freedom of being more imaginative and enabling them to shape up their vision into reality. He is the director of many successful movies including Requiem for a Dream, The Wrestler, and Black Swan and one of his recent projects is based on VR called Spheres. It is a three-part virtual reality black hole series written by Eliza McNitt and produced by Darren Aronofsky's Protozoa Pictures. Aronofsky believes that combining storytelling and VR provides viewers a true emotional experience by taking them to a very convincing and different world. Here are some of the highlights from his interview: How virtual reality based storytelling is different from filmmaking? From a very long time people have been talking about VR replacing films, but it is not going to happen anytime soon. “It may replace how people decide to spend their time but they are two different art forms and most people who work in virtual reality and filmmaking are aware that trying to blend them will not work,” said Aronofsky. Aronofsky feels the experiences created by VR and films are very different. When you are watching a movie you not only watch the character but you also feel how the character is feeling because of empathy. Aronofsky remarks this is a great thing about filmmaking,“It is a great part of filmmaking that you can sit there and you can through close-up enter the subjective experience of the character who takes you on a journey where you are basically experiencing what the character is going through.” In virtual reality, on the other hand, very less character is involved. It is very experiential and instead of being transferred into another person's shoes you are much more yourself. How technology is affecting filmmaking in a better way? One of the biggest breakthroughs enabled by these technologies, according to Aronofsky, is allowing filmmakers to shape their ideas into exactly how they want. He points out that unlike the 70s and 80s, when there were only few “Gandalfs” like Spielberg and George Lucas who were using computers for creating experiences, now computers can be used by anybody to create amazing visual effects, animations, and much more. “Use of computers have unlocked the possibilities of what we can do and what type of stories we can tell,” he added. Technologies such as AI and VR has enabled filmmakers and writers to write and create extremely complicated sequences that otherwise would have taken several human hours. He says, “Machines has given many more ways of looking at the material.” Is there any dark side of using these technologies? Though technology is providing different ways of telling stories, there can be situations where its influence is too much. Aronofsky remarked that there are some filmmakers who have lost control over the use of technology in their films, which has resulted into “visual effects extravaganza”. The huge teams working on these projects focus more on visual effects instead of the storytelling part of filmmaking. But at the same time, there are some filmmakers who know exactly where to draw the line between virtual and reality, giving their audiences beautiful movies to enjoy. “But there are filmmakers like James Cameron who are in control of everything and creating a vision where every single shot is chosen in if it is in virtual setting or in a real setting”, says the moviemaker. On the question of whether AI could replace humans in filmmaking or storytelling, he feels that current technologies are not mature enough to be able to actually understand what the character is feeling. He says, “It’s a terrifying thought… When jokes and humor and stories start to be able to reproduced where you can’t tell the difference between them and the human counterparts is a strange moment… Storytelling is a tricky thing and I am going to be a bit of a Luddite now and put my faith in the constant invention of individuals to do something that a computer won’t.” Does data influences a filmmaker’s decisions? Nowadays every decision is data-driven. Online streaming services tracks each click and swipe to understand user preferences. But, Aronofsky believes that you cannot predict the future even if you have access to so much data. Maybe the popularity of the actors or the locations can help but currently we do not have a fixed formula to predict how much success a film will see. Technologies like AI and VR are helping filmmakers to create visual effects, helping them in digital editing, and all in all have enabled them to put no limits on their imagination. Watch Darren Aronofsky's full talk at Web Summit 2018: https://youtu.be/lkzNZKCxMKc Tim Berners-Lee is on a mission to save the web he invented Web Summit 2018: day 2 highlights UN on Web Summit 2018: How we can create a safe and beneficial digital future for all

Blockstream, a Blockchain-based solution startup, has launched a new Blockchain service called Liquid Network. The Liquid Network is an implementation of an advanced technology called Sidechain. It is a blockchain-based distributed network that provides secure and fast transactions for cryptocurrency traders. What is sidechain? Sidechains are complementary to existing blockchain technology that help in securely transferring the digital entities (assets, tokens, etc.) from one blockchain to other blockchain and vice versa. The sidechain is associated with the main blockchain through a channel that enables the transfer between the two ledgers. The Sidechain then carries additional details of the transaction and thus provides an additional layer of security to the blockchain-based transactions. An important underlying technology associated with Sidechain is “Federation”. A federation is a group that acts as an intermediate to verify all the transactions that happen in Sidechain. Liquid network Liquid network, popularly known as "an inter-exchange settlement network" is built on the top of Bitcoin network. The liquid is based on the concept of Bitcoin sidechain but not exactly sidechain, as it involves more privacy in overseeing the transaction through its network. The liquid network enables a fast transaction by emphasizing on trading mass exchanges through the blockchain ledger. With these functionalities, the Liquid network is said to be bringing in the use of blockchain in production. The main features of the Liquid network are: Liquid Bitcoin (L-BTC): Helps companies provide end-user security for and speedy transfer of Bitcoin with settlements. Issues Assets: This brings Bitcoin features like tokenization, reward points, and attested assets for removing the need for dedicated wallet software. Confidential Transaction technology ensures the privacy of the transfer data by making sure that only transacting parties are overseeing the network. As of its launch on 10 October '18, 23 cryptocurrency companies are using the Liquid network for transactions. The launch of a truly private blockchain network is expected to enable financial institutes in tokenization of various assets like gold, bonds, cryptocurrencies, securities, among others. 9 recommended blockchain online courses JPEG committee wants to apply blockchain to image sharing Google Cloud Launches Blockchain Toolkit to help developers build apps easily

Facial recognition is a contentious technology, to say the least, these days. Yesterday, Axon Enterprises formerly known as Taser International, the largest police body-camera making company in the US announced that it will not incorporate facial-recognition technology in its law-enforcement devices. https://twitter.com/wsisaac/status/1144199471657553920 This move coincides with growing public opposition to facial recognition technology, including from tech workers with some cities in the US mulling to ban its use. Last month, San Francisco became the first city to ban local government use of facial recognition, with Oakland, California, Somerville and Massachusetts, expected to enact similar legislation soon. California's state Legislature is also considering a bill that would ban the use of facial recognition on police body cameras. Axon came to this decision after reviewing a report published by its ethics advisory panel. The panel urged the company not to pair its best-selling body cameras with software that could allow officers to identify people in real time based on their faces. Last year in April, Axon established an AI and Policing Technology Ethics Board. The purpose of the board was to guide and advise the company on ethical issues related to the development and deployment of new artificial intelligence (AI) powered policing technologies. They would advise the company on products which are under consideration or development, and would not formally approve or reject any particular product. This is the first board report that provides thoughtful and actionable recommendations to Axon regarding face recognition technology. The board is an eleven-member external advisory body made up of experts from various fields including AI, computer science, privacy, law enforcement, civil liberties, and public policy. The company also emphasizes on the importance of having a diverse board for the guidance. The current board members are: Ali Farhadi, an Associate Professor in the Department of Computer Science and Engineering at the University of Washington Barry Friedman, an academic and one of the leading authorities on constitutional law, policing, criminal procedure, and federal courts Christy E. Lopez, a Georgetown Law Distinguished Visitor from Practice and former Deputy Chief in the DOJ Civil Rights Division Jeremy Gillula, Tech Projects Director at the Electronic Frontier Foundation Jim Bueermann President of the Police Foundation in Washington, DC Kathleen M. O’Toole, former Chief of Police for the Seattle Police Department Mecole Jordan, Executive Director at United Congress of Community and Religious Organization (UCCRO) Miles Brundage, AI Policy Research Fellow with the Strategic AI Research Center at FHI Tracy Ann Kosa, Senior Program Manager at Google Vera Bumpers, President at National Organization of Black Law Enforcement Executives (NOBLE) Walt McNeil, a Leon County Sheriff in Florida Here are few tweets from some of the board members as well. https://twitter.com/Miles_Brundage/status/1144234344250109952 https://twitter.com/Christy_E_Lopez/status/1144328348040085504   The members of the board cited facial recognition tech's accuracy problems, that it could lead to false identifications, particularly of women and people with dark skin. The technology also could lead to expanded government surveillance and intrusive police activity, the board said. More specifically, the findings of the report are as follows: [box type="shadow" align="" class="" width=""]Facial recognition simply isn’t good enough right now for it to be used ethically. Don’t talk about “accuracy,” talk about specific false negatives and positives, since those are more revealing and relevant. Any facial recognition model that is used shouldn’t be overly customizable, or it will open up the possibility of abuse. Any application of facial recognition should only be initiated with the consent and input of those it will affect. Until there is strong evidence that these programs provide real benefits, there should be no discussion of use. Facial recognition technologies do not exist, nor will they be used, in a political or ethical vacuum, so consider the real world when developing or deploying them.[/box] In a blog post on Axon's website, CEO Rick Smith said current facial recognition technology "raises serious ethical concerns." But Smith also said that his team of artificial intelligence researchers would "continue to evaluate the state of facial recognition technologies," leaving open the possibility of adding the software to body cameras in the future. Axon holds the largest market share among the body cam manufacturer in the United States; it  supplies cameras to 47 of the 60 biggest police agencies. However, it does not say how many police agencies are under the contract, but says that more than 200,000 of its cameras are in use around the country. As per reports from NBC, this move from Axon is appreciated by civil rights and privacy advocates ─ but with skepticism. They noted that real-time facial recognition on police body cameras is not considered feasible at the moment, and they expressed concern that Axon could reverse course once that changed. "This is ultimately an issue about the kind of society we want to live in, not about technical specs," said Harlan Yu, executive director of Upturn, which monitors police agencies' body camera policies, and who is an outspoken Axon critic. https://twitter.com/harlanyu/status/1144278309842370560 Rather than rely on pledges from technology companies, lawmakers should impose regulations on how facial recognition is used, the advocates said. "Axon leaves open the possibility that it may include face recognition in the future, which is why we need federal and state laws ─ like the current proposal in California ─ that would ban the use of facial recognition on body cameras altogether," said Jennifer Lynch, surveillance litigation director at the Electronic Frontier Foundation, a civil liberties nonprofit. Brendan Klare, CEO of Rank One Computing, whose facial recognition software is used by many police departments to identify people in still images, said to NBC that Axon's announcement is a way to make the company look good while making little substantive impact. "The more important thing to point out here is that face recognition on body cameras really isn't technically feasible right now anyways," Klare said. While Axon has very little to lose from its announcement, other players in this industry took this as an opportunity. A couple hours after Axon's announcement, the head of U.K. based company Digital Barriers, trying to break into the U.S. body camera market with its facial recognition-enabled devices ─ tweeted that Axon's move was good news for his company. https://twitter.com/UKZak/status/1144225152915378176 Amazon rejects all 11 shareholder proposals including the employee-led climate resolution at Annual shareholder meeting Amazon patents AI-powered drones to provide ‘surveillance as a service’ San Francisco Board of Supervisors vote in favour to ban use of facial recognition tech in city

ggplot2 team has announced a new version 3.0.0 with breakthrough changes. This new release brings some revolutionary changes within their library to ease advanced data visualizations and create appealing aesthetics. ggplot2 is an open source library in R which allows you to create visual representations. It follows a process of breaking up the advanced graphs into semantic components such as scales and layers. ggplot2 has grown in use considerably within the R community thus becoming one of the popular R packages used today. Some of the noteworthy changes in the library are: Tidy evaluation ggplot2 now supports tidy evaluation. This allows you to easily build plots in the same way you can programmatically build data manipulation pipelines with dplyr Now you can use quasiquotation in aes(), facet_wrap(), and facet_grid() ggplot2 is now more easily programmable and consistent with the rest of the tidyverse packages New features added to the library It supports all simple features using sf with geom_sf() and coord_sf() It can automatically align CRS across layers, draw a graticule, and can set up the correct aspect ratio New stat() function now offers a cleaner and better-documented syntax for calculated aesthetics variables You can use syntax aes(y = stat(count)), thus replacing the old traditional approach of surrounding the variable name with ... (Example - aes(y = ..count..)) A new tag label has been added for identifying plots in addition to title, subtitle and, caption. Layers: geoms, stats, and position adjustments Now you can arrange the horizontal position of plots with variable widths for bars and rectangles in addition to box plots using the new function position_dodge2() There are many other functions and new parameters added to enhanced the layers of the graphics. To know more, you can refer to the GitHub page. Scales and guides Improved support for ordered factors and mapping data/time variables to alpha, size, color, and fill aesthetics, including date_breaks and date_labels arguments Several new functions have been added to make it easy to use Viridis colour scales - scale_colour_viridis_c() and scale_fill_viridis_c() for continuous, and scale_colour_viridis_d() and scale_fill_viridis_d() for discrete To know more about the enhanced support, you can refer the GitHub page. Nonstandard aesthetics Improved support for nonstandard aesthetics. They can now be specified independently of the scale name. There is a huge list of bug fixes and improvements done to the library, if you want to refer to the changes done, you can refer Minor bug fixes and improvements page. You can find the complete list of new updates and changes done to the library along with how to handle common errors and ways to work around them in the breaking changes section of ggplot2 GitHub page. R interface to Python via the Reticulate Package Why choose R for your data mining project 15 Useful Python Libraries to make your Data Science tasks Easier