Tech News - Data

Introducing a new library called Deep Graph Library (DGL) developed by the NYU & AWS teams, Shanghai. DGL is a package built on Python to simplify deep learning on graph, atop of existing deep learning frameworks. DGL is essentially a Python package which serves as an interface between any existing tensor libraries and data that is expressed as graphs. It helps in easy implementation of graph neural networks such as Graph Convolution Networks, TreeLSTM and others. It also maintains high computation efficiency while doing this. This new Python library is made in an effort to make graph implementations in deep learning simpler. According to the results they state, the improvement on some models is as high as 10 times and has better accuracy in some cases. Check out the results on GitHub. Their website states: “We are keen to bring graphs closer to deep learning researchers. We want to make it easy to implement graph neural networks model family. We also want to make the combination of graph based modules and tensor based modules (PyTorch or MXNet) as smooth as possible.” As of now, DGL supports PyTorch v1.0. The autobatching is up to 4x faster than DyNet. DGL is tested on Ubuntu 16.04, macOS X, and Windows 10 and will work on any newer versions of these OSes. Python 3.5 or later is required while Python 3.4 or older is not tested. Support for Python 2 is in the works. Installing it is as same as any other Python package. With pip: pip install dgl And with conda: conda install -c dglteam dgl https://twitter.com/aussetg/status/1072897828677144582 DGL is currently in the beta stage, licensed under Apache 2.0, and they have a Twitter page. You can check out DGL at their website. UK researchers have developed a new PyTorch framework for preserving privacy in deep learning OpenAI launches Spinning Up, a learning resource for potential deep learning practitioners Deep Learning Indaba presents the state of Natural Language Processing in 2018

The Azure team at Microsoft were highly excited to release the general availability of Azure SQL Data Sync tool for synchronization with their on-premises databases. This new tool allows database administrators to synchronize the data access between Azure SQL Database and any other SQL hosted server or local servers, both unidirectionally and bidirectionally. This new data sync tool allows you to distribute your data apps globally with a local replication available in each region, keeping data synchronization continuous across all the regions. This tool would help to significantly eradicate the connection failure and eliminate the issues related to network latency. It will also boost the response time of the applications and enhance the reliability of the application run time. Features/Capabilities of Azure SQL data Sync: Easy-to-Config - Simple and better configuration of database workflow with exciting user experience Speedy and reliable database schema refresh - Faster loading of database schemas with new Server Management Objects (SMO) library Security for Data Sync - End-to-end encryption provided for both unidirectional and bi-directional data flows with GDPR compliance. However, this particular tool would not be a true friend to DBAs as it does not support disaster recovery task. Microsoft has also made it very clear that this technology would not be supporting scaling Azure workloads, nor the Azure’s Database Migration Service. Check out the Azure SQL Data Sync Setup documentation to get started. To know more details, you can refer to the official announcement at official Microsoft web page. Get SQL Server user management right Top 10 MySQL 8 performance benchmarking aspects to know Data Exploration using Spark SQL

Two days ago, Google astonished many people around the world with claims that they have achieved a major milestone in quantum computing, one which has largely been an unattainable feat until now. In a paper titled “Quantum supremacy using a programmable superconducting processor”, Google explains how its 53-bit quantum computer named ‘Sycamore’ took only 200 seconds to perform a sensitive computation that would otherwise take the world's fastest supercomputer 10,000 years. This, Google claims is their attainment of ‘quantum supremacy’. If confirmed, this would be the first major milestone in harnessing the principles of quantum mechanics to solve computational problems. Google’s AI Quantum team and John Martinis, physicist at the University of California are the prime contributors to this achievement. NASA Ames Research Center, Oak Ridge National Laboratory and Forschungszentrum Jülich have also helped Google in implementing this experiment. In quantum computing, quantum supremacy is the potential ability of any device to solve problems that classical computers practically cannot. According to Sundar Pichai, Google CEO, “For those of us working in science and technology, it’s the “hello world” moment we’ve been waiting for—the most meaningful milestone to date in the quest to make quantum computing a reality.” This announcement from Google comes exactly one month after the same paper was leaked online. However, following Google’s announcement, IBM is arguing “an ideal simulation of the same task can be performed on a classical system in 2.5 days with far greater fidelity.” According to IBM, as proposed by John Preskill in 2012, the original meaning of the term “quantum supremacy,” is the point where quantum computers can do things that classical computers can’t. Since, Google has not yet achieved this threshold, IBM argues that their claims are wrong. IBM says that in the published paper, Google has assumed that the RAM storage requirements in a traditional computer would be massive. However, if a different approach of using both RAM and hard drive space to store and manipulate the state vector is employed, the 10,000 years specified by Google will drop considerably. Thus, IBM refutes Google’s claims and stated that in its strictest definition, the quantum supremacy standard has not been met by anybody until now. IBM believes that “fundamentally, quantum computers will never reign “supreme” over classical computers, but will rather work in concert with them, since each have their unique strengths.”  IBM further added that the term ‘supremacy’ is currently being misunderstood and urged everybody in the community to treat Google’s claims “with a large dose of skepticism.” Read More: Has IBM edged past Google in the battle for Quantum Supremacy? Though Google has not directly responded to IBM’s accusations, in a statement to Forbes, a Google spokesperson said, “We welcome ideas from the research community on new applications that work only on NISQ-era processors like Sycamore and its successors. We’ve published our circuits so the whole community can explore this new space. We’re excited by what’s possible now that we have this unique new resource.” Although IBM is skeptical of Google’s claims, the news of Google’s accomplishment is making waves all around the world. https://twitter.com/rhhackett/status/1186949190695313409 https://twitter.com/nasirdaniya/status/1187152799055929346 Google’s experiment with the Sycamore processor To achieve this milestone, Google researchers developed Sycamore, the high-fidelity quantum logic gates processor consisting of a two-dimensional array of 54 transmon qubits. Sycamore consists of a two-dimensional grid where each qubit is connected to four other qubits. Each qubit in the processor is tunably coupled to four nearest neighbors, in a rectangular lattice and are forward-compatible for error correction. As a result, the chip has enough connectivity to let  the qubit states quickly interact throughout the entire processor. This feature of Sycamore is what makes it distinct from a classical computer. Image Source: Google blog In the Sycamore quantum processor, “Each run of a random quantum circuit on a quantum computer produces a bitstring, for example 0000101. Owing to quantum interference, some bit strings are much more likely to occur than others when we repeat the experiment many times. However, finding the most likely bit strings for a random quantum circuit on a classical computer becomes exponentially more difficult as the number of qubits (width) and number of gate cycles (depth) grow,” states John Martinis, Chief Scientist Quantum Hardware and Sergio Boixo, Chief Scientist Quantum Computing Theory, Google AI Quantum. Image Source: Google blog For the experiment, the Google researchers ran a random simplified circuit from 12 to 53 qubits with the circuit depth kept constant. Next, they checked the performance of the quantum computer using classical simulations and compared the performance of a quantum computer with a theoretical model. After verifying that the quantum system was working, the researchers ran a random hard circuit with 53 qubits and this time, allowed the circuit depth to expand until the point where the classical simulation became infeasible. At the end, it was found that this quantum computation cannot be emulated on a classical computer and hence, this opens “a new realm of computing to be explored,” says Google. The Google team is now working on quantum supremacy applications like quantum physics simulation, quantum chemistry, generative machine learning, and more. After procuring “certifiable quantum randomness”, Google is now working on testing this algorithm to develop a prototype that can provide certifiable random numbers. Leaving IBM’s accusation aside, many people are excited about Google’s great achievement. https://twitter.com/christina_dills/status/1187074109550800897 https://twitter.com/Inzilya777/status/1187102111429021696 Few people believe that Google is making a hullabaloo of a not-so-successful experiment. A user on Hacker News comments, “Summary: - Google overhyped their results against a weak baseline. This seems to be commonplace in academic publishing, especially new-ish fields where benchmarks aren't well-established. There was a similar backlash against OpenAI's robot hand, where they used simulation for the physical robotic movements and used a well-known algorithm for the actual Rubik's Cube solving. I still think it's an impressive step forward for the field.” Check out the video of Google’s demonstration of quantum supremacy below. https://youtu.be/-ZNEzzDcllU Smart Spies attack: Alexa and Google Assistant can eavesdrop or vish (voice phish) unsuspecting users, disclose researchers from SRLabs Made by Google 2019: Google’s hardware event unveils Pixel 4 and announces the launch date of Google Stadia After backlash for rejecting a uBlock Origin update from the Chrome Web Store, Google accepts ad-blocking extension

The Apache Flink community released its 1.6.0 version yesterday. Apache Flink 1.6.0 release is the seventh major release in the 1.x.y series. This Flink version is API-compatible with the previous 1.x.y releases for APIs annotated with the @Public annotation. Apache Flink 1.6.0 enables users to seamlessly run fast data processing and also build data-driven, data-intensive applications effortlessly. Features and Improvements in Apache Flink 1.6.0 In this version, the Flink community has added a Jepsen based test suite (FLINK-9004). This suite validates the behavior of Flink’s distributed cluster components under real-world faults. It is the community’s first step towards a higher test coverage for Flink’s fault tolerance mechanisms. The other major features include, An improved State Support for Flink The support for State TTL feature allows one to specify a time-to-live (TTL) for Flink state. One the TTL exceeds, Flink will no longer give access to the respective state values. The expired data is cleaned up on access such that the operator keyed state doesn’t grow infinitely and it won’t be included in subsequent checkpoints. This feature fully complies with new data protection regulations (e.g. GDPR). With the scalable Timers Based on RocksDB, Flink’s timer state can now be stored in RocksDB, allowing the technology to support significantly bigger timer state since it can go out of core/spill to disk. One can perform fast timer deletions with Flink’s improvised internal timer data structure such that the deletion complexity is reduced from O(n) to O(log n). This significantly improves Flink jobs using timers. Extended Deployment Options in Flink 1.6.0 Flink 1.6.0 provides an easy-to-use container entrypoint to bootstrap a job cluster. Combining this entrypoint with a user-code jar creates a self-contained image which automatically executes the contained Flink job when deployed. With a fully RESTified job submission, the Flink client can now send all job-relevant content via a single POST call to the server. This allows a much easier integration with cluster management frameworks and container environments since opening custom ports is no longer necessary. SQL and Table API enhancements The SQL Client CLI now supports the registration of user-defined functions, which improves the CLI’s expressiveness. This is because SQL queries can be enriched with more powerful custom table, aggregate, and scalar functions. The Apache Flink 1.6.0 now supports Batch Queries in SQL Client CLI,  INSERT INTO Statements in SQL Client CLI, and SQL Avro. Table sinks can now be defined in a YAML file using string-based properties without having to write a single line of code, in this release. New Kafka Table Sink uses the new unified APIs and supports both JSON and Avro formats. Improved Expressiveness of SQL and Table API where SQL aggregate functions support the DISTINCT keyword. Queries such as COUNT(DISTINCT column) are supported for windowed and non-windowed aggregations. Both SQL and Table API now include more built-in functions such as MD5, SHA1, SHA2, LOG, and UNNEST for multisets. Hardened CEP Library The CEP operator’s internal NFA state is now backed by Flink state supporting larger use cases. More Expressive DataStream Joins Flink 1.6.0 adds support for interval joins in the DataStream API. With this feature it is now possible to join together events from different streams to each other. Intra-Cluster Mutual Authentication Flink’s cluster components now enforce mutual authentication with their peers. This allows only Flink components to talk to each other, making it difficult for malicious actors to impersonate Flink components in order to eavesdrop on the cluster communication. Read more about this release in detail in Apache Flink 1.6.0 release notes. Implementing fault-tolerance in Spark Streaming data processing applications with Apache Kafka How to get started with Azure Stream Analytics and 7 reasons to choose it Performing Vehicle Telemetry job analysis with Azure Stream Analytics tools  

Google has made it quite accessible for people to collaborate their documents, spreadsheets, and so on, with the Google Drive feature. What next? If you are one of those data science nerds who love coding, this roll-out from Google would be an amazing experimental ground for you. Google released its coLaboratory project, a new tool, and a boon for data science and analysis. It is designed in a way to make collaborating on data easier; similar to a Google document. This means it is capable of running code and providing simultaneous output within the document itself. Collaboration is what sets coLaboratory apart. It allows an improved collaboration among people having distinct skill sets--one may be great at coding, while the other might be well aware of the front-end or GUI aspects of the project. Just as you store and share a Google document or spreadsheets, you can store and share code with coLaboratory notebooks, in Google Drive. All you have to do is, click on the 'Share' option at the top right of any coLaboratory notebook. You can also look up to the Google Drive file sharing instructions. Thus, it sets new improvements for the ad-hoc workflows without the need of mailing documents back and forth. CoLaboratory includes a Jupyter notebook environment that does not require any setup for using it. With this, one does not need to download, install, or run anything on their computer. All they would need is, just a browser and they can use and share Jupyter notebooks. At present, coLaboratory functions with Python 2.7 on the desktop version of Chrome only. The reason for this is, coLab with Python 2.7 has been an internal tool for Google, for many years. Although, making it available on other browsers and with an added support for other Jupyter Kernels such as R or Scala is on the cards, soon. CoLaboratory’s GitHub repository contains two dependent tools, which one can make use of to leverage the tool onto the browser. First is the coLaboratory Chrome App and the other is coLaboratory with Classic Jupyter Kernels.  Both tools can be used for creating and storing notebooks within Google Drive. This allows a collaborative editing within the notebooks. The only difference is that Chrome App executes all the code within its browser using the PNaCl Sandbox. Whereas, the CoLaboratory classic code execution is done using the local Jupyter kernels (IPython kernel) that have a complete access to the host systems and files. The coLaboratory Chrome App aids in setting up a collaborative environment for data analysis. This can be a hurdle at times, as requirements vary among different machines and operating systems. Also, the installation errors can be cryptic too. However, just with a single click, coLaboratory, IPython and a large set of popular scientific python libraries can be installed. Also, because of the Portable Native Client (PNaCl), coLaboratory is secure and runs at local speeds. This allows new users to set out on exploring IPython at a faster speed. Here’s what coLaboratory brings about for the code-lovers: No additional installation required the browser does it all The capabilities of coding now within a document Storing and sharing the notebooks on Google Drive Real-time collaboration possible; no fuss of mailing documents to and fro You can find a detailed explanation of the tool on GitHub.  

Edge services and edge computing have been in talks since at least the 90s. When Edge computing is extended to the cloud it can be managed and consumed as if it were local infrastructure. The logic is simple. It’s the same as how humans find it hard to interact with infrastructure that is too far away. Edge Analytics is the exciting area of data analytics that is gaining a lot of attention these days. While traditional analytics, answer questions like what happened, why it happened, what is likely to happen and options on what you should do about it Edge analytics is data analytics in real time. It deals with the operations performed on data at the edge of a network either at or close to a sensor, a network switch or some other connected device. This saves time and overhead issues as well as latency problems. As they rightly say, Time is money! Now imagine using AI to facilitate edge analytics. What does AI in Edge Computing mean When Edge computing is extended to the cloud it can be managed and consumed as if it were local infrastructure. The logic is simple. It’s the same as how humans find it hard to interact with infrastructure that is too far away. Smart Applications these rely on sending tons of information to the cloud. Data can be compromised in such situations. As such, security and privacy challenges may arise. Application developers will have to consider whether the bulk of information sent to the cloud contains personally identifiable information (PII) and whether storing it is in breach of privacy laws. They’ll also have to take the necessary measures to secure the information they store and prevent it from being stolen, or accessed or shared illegally. Now that is a lot of work! Enter  “Intelligent Edge” computing used to save the day!  Edge computing by itself will not replace the power of the cloud. It can, however, reduce cloud payloads drastically when used in collaboration with machine learning. transform the AI’s operation model into that of the human brain: perform routine and time-critical decisions at the edge and only refer to the cloud where more intensive computation and historical analysis is needed. Why use AI in edge computing Most mobile apps, IoT devices and other applications that work with AI and machine learning algorithms and applications rely on the processing power of the cloud or on a datacenter situated thousands of miles away. They have little or no intelligence to apply processing at the edge. Even if you show your favorite pet picture to your smart device a thousand times, it’ll still have to look it up in its cloud server in order to recognize if its a dog or a cat for the 1001st time. OK, who cares if it takes a couple of minutes more for my device to differentiate between a dog and a cat! Let’s consider a robot surgeon which wants to perform a sensitive operation on a patient. It will need to be able to analyze images and make decisions dozens of times per second. The round trip to the cloud would cause lags that could have severe consequences. God forbid, if there is a cloud outage or poor internet connectivity. To perform this task efficiently, faster and to reduce the back and forth communication involved between the cloud and the device, implementing AI in edge is a good idea. Top 7 AI for edge computing use cases that caught our attention Now that you are convinced that Intelligent edge or AI powered edge computing does have potential, here are some recent advancements in edge AI and some ways it is being used in the real world. #1 Consumer IoT: Microsoft’s $5 billion investment in IoT to empower the intelligent cloud and the intelligent edge One of the central design principles of Microsoft’s intelligent edge products and services is to secure data no matter where it is stored. Azure Sphere is one of their intelligent edge solutions to power and protect connected microcontroller unit (MCU)-powered devices. There are 9 billion MCU-powered devices shipping every year, which power everything from household stoves and refrigerators to industrial equipment. That’s intelligent edge for you on the consumer end of the application spectrum. Let’s look at the industrial application use case next. #2  Industrial IoT: GE adds edge analytics, AI capabilities to its industrial IoT suite To make a mark in the field of industrial internet of things (IIoT), GE Digital is adding features to its Predix platform as a service (PaaS). This will let industrial enterprises run predictive analytics as close as possible to data sources, whether they be pumps, valves, heat exchangers, turbines or even machines on the move. The main idea behind edge computing is to analyze data in near real-time, optimize network traffic and cut costs. GE Digital has been working to integrate the company's field service management (FSM) software with GE products and third-party tools. For example, artificial intelligence-enabled predictive analytics now integrate the Apache Spark AI engine to improve service time estimates. New application integration features let service providers launch and share FSM data with third-party mobile applications installed on the same device. Read the whole story on Network World. #3 Embedded computing and robotics: Defining (artificial) intelligence at the edge for IoT systems Machine intelligence has largely been the domain of computer vision (CV) applications such as object recognition. While artificial intelligence technology is thus far still in its infancy, its benefits for advanced driver assistance systems (ADAS), collaborative robots (cobots), sense-and-avoid drones, and a host of other embedded applications are obvious. Related to the origins of AI technology is the fact that most, if not all, machine learning frameworks were developed to run on data center infrastructure. As a result, the software and tools required to create CNNs/DNNs for embedded targets have been lacking. In the embedded machine learning sense, this has meant that intricate knowledge of both embedded processing platforms and neural network creation has been a prerequisite for bringing AI to the embedded edge – a luxury most organizations do not have or is extremely time-consuming if they do. Thanks to embedded silicon vendors, this paradigm is set to shift. Based on the power consumption benchmarks, AI technology is quickly approaching deeply embedded levels. Read the whole article on embedded computing design to know more about how Intelligent edge is changing our outlook towards embedded systems. #4 Smart grids: Grid Edge Control and Analytics Grid Edge Controllers are intelligent servers, deployed as an interface between the edge nodes and the utility’s core network. Smart Grid, as we know, is essentially the concept of establishing a two-way communication between distribution infrastructure, consumer and the utility head end using Internet Protocol. From residential rooftops to solar farms, commercial solar, electric vehicles and wind farms, smart meters are generating a ton of data. This helps utilities to view the amount of energy available and required, allowing their demand response to become more efficient, avoid peaks and reduce costs. This data is first processed in the Grid Edge Controllers that perform local computation and analysis of the data, only sending necessary actionable information over a wireless network to the Utility. #5 Predictive maintenance: Oil and Gas Remote Monitoring Using Internet of Things devices such as temperature, humidity, pressure, and moisture sensors, alongside internet protocol (IP) cameras and other technologies, oil and gas monitoring operations produce an immense amount of data which provide key insights into the health of their specific systems. Edge computing allows this data to be analysed, processed, and then delivered to end-users in real-time. This, in turn, enables control centers to access data as it occurs in order to foresee and prevent malfunctions or incidents before they occur. #6 Cloudless Autonomous Vehicles Self-driving cars and intelligent traffic management systems are already the talk of the town today and the integration of edge AI could be the next big step. When it comes to autonomous systems, safety is paramount. Any delay, malfunction, or anomaly within autonomous vehicles can prove to be fatal. Calculating a number of parameters at the same time, edge computing and AI enables safe and fast transportation with quick decision making capabilities. #7 Intelligent Traffic Management Edge computing is able to analyse and process data on the traffic hardware itself and finds ways to remove unnecessary traffic. This reduces the overall amount of data that needs to be transmitted across a given network and helps to reduce both operating and storage costs. What’s next for AI enabled edge computing? The intelligent edge will allow humans to simplify multi-faceted processes by replacing the manual process of sorting and identifying complex data, key insights and actionable plans. This forte of technology can help humans gain a competitive edge by having better decision-making, improved ROI, operational efficiency and cost savings. However, on the flip side, there are also many cons to machine learning based edge computing.. The cost of deploying and managing an edge will be considerably huge. With all rapidly evolving technologies- evaluating, deploying and operating edge computing solutions has its risks. A key risk area being -security. Tons of data needs to be made available for processing at the edge and where there is data, there is always a fear of data breach. Performing so many operations on the data also can be challenging. All-in-All even though the concept of incorporating AI into edge computing is exciting, some work does need to be done to get intelligent edge-based solutions l fully set up, functional and running smoothly in production. What’s your take on this digital transformation? Reinforcement learning model optimizes brain cancer treatment Tesla is building its own AI hardware for self-driving cars OpenAI builds reinforcement learning based system giving robots human like dexterity

PyTorch 0.3.0 has removed stochastic functions, i.e. Variable.reinforce(), citing “limited functionality and broad performance implications.” The Python package has added a number of performance improvements, new layers, support to ONNX, CUDA 9, cuDNN 7, and “lots of bug fixes” in the new version. “The motivation for stochastic functions was to avoid book-keeping of sampled values. In practice, users were still book-keeping in their code for various reasons. We constructed an alternative, equally effective API, but did not have a reasonable deprecation path to the new API. Hence this removal is a breaking change,” PyTorch team said. To replace stochastic functions, they have introduced the torch.distributions package. So if your previous code looked like this: probs = policy_network(state) action = probs.multinomial() next_state, reward = env.step(action) action.reinforce(reward) action.backward() This could be the new equivalent code: probs = policy_network(state) # NOTE: categorical is equivalent to what used to be called multinomial m = torch.distributions.Categorical(probs) action = m.sample() next_state, reward = env.step(action) loss = -m.log_prob(action) * reward loss.backward()   What is new in PyTorch 0.3.0?   Unreduced losses Now, Some loss functions can compute per-sample losses in a mini-batch By default PyTorch sums losses over the mini-batch and returns a single scalar loss. This was limiting to users. Now, a subset of loss functions allow specifying reduce=False to return individual losses for each sample in the mini-batch Example: loss = nn.CrossEntropyLoss(..., reduce=False) Currently supported losses: MSELoss, NLLLoss, NLLLoss2d, KLDivLoss, CrossEntropyLoss, SmoothL1Loss, L1Loss More loss functions will be covered in the next release An in-built Profiler in the autograd engine PyTorch has built a low-level profiler to help you identify bottlenecks in your models. Let us start with an example: >>> x = Variable(torch.randn(1, 1), requires_grad=True) >>> with torch.autograd.profiler.profile() as prof: ... y = x ** 2 ... y.backward() >>> # NOTE: some columns were removed for brevity ... print(prof) -------------------------------- ---------- --------- Name CPU time CUDA time ------------------------------- ---------- --------- PowConstant 142.036us 0.000us N5torch8autograd9GraphRootE 63.524us 0.000us PowConstantBackward 184.228us 0.000us MulConstant 50.288us 0.000us PowConstant 28.439us 0.000us Mul 20.154us 0.000us N5torch8autograd14AccumulateGradE 13.790us 0.000us N5torch8autograd5CloneE 4.088us 0.000us The profiler works for both CPU and CUDA models. For CUDA models, you have to run your python program with a special nvprof prefix. For example: nvprof --profile-from-start off -o trace_name.prof -- python <your arguments> # in python >>> with torch.cuda.profiler.profile(): ... model(x) # Warmup CUDA memory allocator and profiler ... with torch.autograd.profiler.emit_nvtx(): ... model(x) Then, you can load trace_name.prof in PyTorch and print a summary profile report. >>> prof = torch.autograd.profiler.load_nvprof('trace_name.prof') >>> print(prof) For additional documentation, you can visit this link. Higher order gradients v0.3.0 has added higher-order gradients support for the following layers: ConvTranspose, AvgPool1d, AvgPool2d, LPPool2d, AvgPool3d, MaxPool1d, MaxPool2d, AdaptiveMaxPool, AdaptiveAvgPool, FractionalMaxPool2d, MaxUnpool1d, MaxUnpool2d, nn.Upsample, ReplicationPad2d, ReplicationPad3d, ReflectionPad2d PReLU, HardTanh, L1Loss, SoftSign, ELU, RReLU, Hardshrink, Softplus, SoftShrink, LogSigmoid, Softmin, GLU MSELoss, SmoothL1Loss, KLDivLoss, HingeEmbeddingLoss, SoftMarginLoss, MarginRankingLoss, CrossEntropyLoss DataParallel Optimizers optim.SparseAdam: Implements a lazy version of Adam algorithm suitable for sparse tensors. (In this variant, only moments that show up in the gradient get updated, and only those portions of the gradient get applied to the parameters.) Optimizers now have an add_param_group function that lets you add new parameter groups to an already constructed optimizer. New layers and nn functionality Added AdpativeMaxPool3d and AdaptiveAvgPool3d Added LPPool1d F.pad now has support for: 'reflection' and 'replication' padding on 1d, 2d, 3d signals (so 3D, 4D and 5D Tensors) constant padding on n-d signals nn.Upsample now works for 1D signals (i.e. B x C x L Tensors) in nearest and linear modes. Allow user to not specify certain input dimensions for AdaptivePool*d and infer them at runtime. For example: # target output size of 10x7 m = nn.AdaptiveMaxPool2d((None, 7)) DataParallel container on CPU is now a no-op (instead of erroring out) New Tensor functions and features Introduced torch.erf and torch.erfinv that compute the error function and the inverse error function of each element in the Tensor. Adds broadcasting support to bitwise operators Added Tensor.put_ and torch.take similar to numpy.take and numpy.put. The take function allows you to linearly index into a tensor without viewing it as a 1D tensor first. The output has the same shape as the indices. The put function copies value into a tensor also using linear indices. Adds zeros and zeros_like for sparse Tensors. 1-element Tensors can now be casted to Python scalars. For example: int(torch.Tensor([5]))works now. Other additions Added torch.cuda.get_device_name and torch.cuda.get_device_capability that do what the names say. Example: >>> torch.cuda.get_device_name(0) 'Quadro GP100' >>> torch.cuda.get_device_capability(0) (6, 0) If one sets torch.backends.cudnn.deterministic = True, then the CuDNN convolutions use deterministic algorithms torch.cuda_get_rng_state_all and torch.cuda_set_rng_state_all are introduced to let you save / load the state of the random number generator over all GPUs at once torch.cuda.emptyCache() frees the cached memory blocks in PyTorch's caching allocator. This is useful when having long-running ipython notebooks while sharing the GPU with other processes. API changes softmax and log_softmax now take a dim argument that specifies the dimension in which slices are taken for the softmax operation. dim allows negative dimensions as well (dim = -1 will be the last dimension) torch.potrf (Cholesky decomposition) is now differentiable and defined on Variable Remove all instances of device_id and replace it with device, to make things consistent torch.autograd.grad now allows you to specify inputs that are unused in the autograd graph if you use allow_unused=True This gets useful when using torch.autograd.grad in large graphs with lists of inputs / outputs For example: x, y = Variable(...), Variable(...) torch.autograd.grad(x * 2, [x, y]) # errors torch.autograd.grad(x * 2, [x, y], allow_unused=True) # works pad_packed_sequence now allows a padding_value argument that can be used instead of zero-padding Dataset now has a + operator (which uses ConcatDataset). You can do something like MNIST(...) + FashionMNIST(...) for example, and you will get a concatenated dataset containing samples from both. torch.distributed.recv allows Tensors to be received from any sender (hence, src is optional). recv returns the rank of the sender. adds zero_() to Variable Variable.shape returns the size of the Tensor (now made consistent with Tensor) torch.version.cuda specifies the CUDA version that PyTorch was compiled with Added a missing function random_ for CUDA. torch.load and torch.save can now take a pathlib.Path object, which is a standard Python3 typed filepath object If you want to load a model's state_dict into another model (for example to fine-tune a pre-trained network), load_state_dict was strict on matching the key names of the parameters. Now Pytorch provides a strict=False option to load_state_dict where it only loads in parameters where the keys match, and ignores the other parameter keys. added nn.functional.embedding_bag that is equivalent to nn.EmbeddingBag Performance Improvements The overhead of torch functions on Variables was around 10 microseconds. This has been brought down to ~1.5 microseconds by moving most of the core autograd formulas into C++ using ATen library. softmax and log_softmax are now 4x to 256x faster on the GPU after rewriting the gpu kernels 2.5x to 3x performance improvement of the distributed AllReduce (gloo backend) by enabling GPUDirect nn.Embedding's renorm option is much faster on the GPU. For embedding dimensions of 100k x 128 and a batch size of 1024, it is 33x faster. All pointwise ops now use OpenMP and get multi-core CPU benefits Added a single-argument version of torch.arange. For example torch.arange(10) Framework Interoperability DLPack Interoperability DLPack Tensors are cross-framework Tensor formats. We now have torch.utils.to_dlpack(x) and torch.utils.from_dlpack(x) to convert between DLPack and torch Tensor formats. The conversion has zero memory copy and hence is very efficient. Model exporter to ONNX ONNX is a common model interchange format that can be executed in Caffe2, CoreML, CNTK, MXNet, and Tensorflow at the moment. PyTorch models that are ConvNet-like and RNN-like (static graphs) can now be shipped to the ONNX format. There is a new module torch.onnx (http://pytorch.org/docs/0.3.0/onnx.html) which provides the API for exporting ONNX models. The operations supported in this release are: add, sub (nonzero alpha not supported), mul, div, cat, mm, addmm, neg, tanh, sigmoid, mean, t, transpose, view, split, squeeze expand (only when used before a broadcasting ONNX operator; e.g., add) prelu (single weight shared among input channels not supported) threshold (non-zero threshold/non-zero value not supported) Conv, ConvTranspose, BatchNorm, MaxPool, RNN, Dropout, ConstantPadNd, Negate elu, leaky_relu, glu, softmax, log_softmax, avg_pool2d unfold (experimental support with ATen-Caffe2 integration) Embedding (no optional arguments supported) RNN FeatureDropout (training mode not supported) Index (constant integer and tuple indices supported) Usability Improvements More cogent error messages during indexing of Tensors / Variables Breaking changes Add proper error message for specifying dimension on a tensor with no dimensions better error messages for Conv*d input shape checking More user-friendly error messages for LongTensor indexing Better error messages and argument checking for Conv*d routines Trying to construct a Tensor from a Variable fails more appropriately If you are using a PyTorch binary with insufficient CUDA version, then a warning is printed to the user. Fixed incoherent error messages in load_state_dict Fix error message for type mismatches with sparse tensors Bug fixes torch Fix CUDA lazy initialization to not trigger on calls to torch.manual_seed (instead, the calls are queued and run when CUDA is initialized) Tensor if x is 2D, x[[0, 3],] was needed to trigger advanced indexing. The trailing comma is no longer needed, and you can do x[[0, 3]] x.sort(descending=True) used to incorrectly fail for Tensors. Fixed a bug in the argument checking logic to allow this. Tensor constructors with numpy input: torch.DoubleTensor(np.array([0,1,2], dtype=np.float32)) torch will now copy the contents of the array in a storage of appropriate type. If types match, it will share the underlying array (no-copy), with equivalent semantics to initializing a tensor with another tensor. On CUDA, torch.cuda.FloatTensor(np.random.rand(10,2).astype(np.float32)) will now work by making a copy. ones_like and zeros_like now create Tensors on the same device as the original Tensor expand and expand_as allow expanding an empty Tensor to another empty Tensor torch.HalfTensor supports numpy() and torch.from_numpy Added additional size checking for torch.scatter Fixed random_ on CPU (which previously had a max value of 2^32) for DoubleTensor and LongTensor Fix ZeroDivisionError: float division by zero when printing certain Tensors torch.gels when m > n had a truncation bug on the CPU and returned incorrect results. Fixed. Added a check in tensor.numpy() that checks if no positional arguments are passed Before a Tensor is moved to CUDA pinned memory, added a check to ensure that it is contiguous Fix symeig on CUDA for large matrices. The bug is that not enough space was being allocated for the workspace, causing some undefined behavior. Improved the numerical stability of torch.var and torch.std by using Welford's algorithm The Random Number Generator returned uniform samples with inconsistent bounds (inconsistency in cpu implementation and running into a cublas bug). Now, all uniform sampled numbers will return within the bounds [0, 1), across all types and devices Fixed torch.svd to not segfault on large CUDA Tensors (fixed an overflow error in the magma bindings) Allows empty index Tensor for index_select (instead of erroring out) Previously when eigenvector=False, symeig returned some unknown value for the eigenvectors. Now this is corrected. sparse Fix bug with 'coalesced' calculation in sparse 'cadd' Fixes .type() not converting indices tensor. Fixes sparse tensor coalesce on the GPU in corner cases autograd Fixed crashes when calling backwards on leaf variable with requires_grad=False fix bug on Variable type() around non-default GPU input. when torch.norm returned 0.0, the gradient was NaN. We now use the subgradient at 0.0, so the gradient is 0.0. Fix an correctness issue with advanced indexing and higher-order gradients torch.prod's backward was failing on the GPU due to a type error, fixed. Advanced Indexing on Variables now allows the index to be a LongTensor backed Variable Variable.cuda() and Tensor.cuda() are consistent in kwargs options optim torch.optim.lr_scheduler is now imported by default. nn Returning a dictionary from a nn.Module's forward function is now supported (used to throw an error) When register_buffer("foo", ...) is called, and self.foo already exists, then instead of silently failing, now raises a KeyError Fixed loading of older checkpoints of RNN/LSTM which were missing _data_ptrs attributes. nn.Embedding had a hard error when using the max_norm option. This is fixed now. when using the max_norm option, the passed-in indices are written upon (by the underlying implementation). To fix this, pass a clone of the indices to the renorm kernel. F.affine_grid now can take non-contiguous inputs EmbeddingBag can accept both 1D and 2D inputs now. Workaround a CuDNN bug where batch sizes greater than 131070 fail in CuDNN BatchNorm fix nn.init.orthogonal to correctly return orthonormal vectors when rows < cols if BatchNorm has only 1 value per channel in total, raise an error in training mode. Make cuDNN bindings respect the current cuda stream (previously raised incoherent error) fix grid_sample backward when gradOutput is a zero-strided Tensor Fix a segmentation fault when reflection padding is out of Tensor bounds. If LogSoftmax has only 1 element, -inf was returned. Now this correctly returns 0.0 Fix pack_padded_sequence to accept inputs of arbitrary sizes (not just 3D inputs) Fixed ELU higher order gradients when applied in-place Prevent numerical issues with poisson_nll_loss when log_input=False by adding a small epsilon distributed and multi-gpu Allow kwargs-only inputs to DataParallel. This used to fail: n = nn.DataParallel(Net()); out = n(input=i) DistributedDataParallel calculates num_samples correctly in python2 Fix the case of DistributedDataParallel when 1-GPU per process is used. Allow some params to be requires_grad=False in DistributedDataParallel Fixed DataParallel to specify GPUs that don't include GPU-0 DistributedDataParallel's exit doesn't error out anymore, the daemon flag is set. Fix a bug in DistributedDataParallel in the case when model has no buffers (previously raised incoherent error) Fix __get_state__ to be functional in DistributedDataParallel (was returning nothing) Fix a deadlock in the NCCL bindings when GIL and CudaFreeMutex were starving each other Among other fixes,model.zoo.load_url now first attempts to use the requests library if available, and then falls back to urllib. To download the source code, click here.

Microsoft announced the March release of Azure Data Studio, yesterday. This latest Azure Data Studio release explores features such as preview support for PostgreSQL in Azure Data Studio,  corresponding preview PostgreSQL extension in Visual Studio Code (VS Code), and SQL Notebooks, among others. What’s new in Azure Data Studio? PostgreSQL extension for Azure Data Studio There’s new preview support for PostgreSQL in Azure Data Studio. The preview support adds on-premises and Azure Database for PostgreSQL to its existing support for SQL Server, Azure SQL Database, Azure SQL Managed Instance, Azure SQL Data Warehouse, and SQL Server 2019 big data clusters. The Azure Data Studio extension for PostgreSQL comprises a Tools API service that offers data management and high performant query execution capabilities. Azure Data Studio also provides a modern, keyboard-focused PostgreSQL coding experience, that simplifies your everyday tasks. Users can also run on-demand SQL queries now such as the view and save results as text, JSON, or Excel. There’s also an extension marketplace for Azure Data Studio for developers that help them build and contribute back into the open source ecosystem. Microsoft team states that it is making the new PostgreSQL extension experience open source under the MIT license. This will allow the users to connect to all PostgreSQL databases in case they’re running on Azure (Azure Database for PostgreSQL). SQL Notebooks Using SQL Notebooks, you can easily interleave the written instructions, analysis, diagrams, and the animated GIFs using markdown. You can then add code cells with the SQL code to be executed. The SQL Notebook functionality has been built into the base of Azure Data Studio product and requires no additional extensions to connect to servers and execute the SQL result sets. These SQL notebooks can be used like any other regular query editor. You can get started with SQL notebooks just like a regular query editor. In case you’d like to use other languages such as Python, R, or Scala, you’ll be prompted to install other additional dependencies. PowerShell extension PowerShell extension from Virtual Studio (VS) Code is now featured in the Azure Data Studio marketplace. The new PowerShell extension aligns with the other automation scenarios used by the database administrators and developers. There’s an integrated terminal in Azure Data Studio that makes it easy for users to integrate PowerShell experiences with data. SQL Server dacpac extension Microsoft team mentions that it has been trying to improve the Data-Tier Application Wizard in Azure Data Studio after receiving feedback from the community. Originally shipped with the SQL Server Import extension, this feature will now be shipped as a separate extension. This is because the team plans to bring more features making it easy to use dacpacs and bacpacs in Azure Data Studio. This extension will also be included in the Admin pack for SQL Server, an extension pack that lets you quickly download the popular features from SQL Server Management Studio. Other Changes Community extension highlight: Queryplan.show The Queryplan.show extension adds integration support to visualize query plans using the community extension Queryplan.show. Visual Studio Code Refresh from 1.26.1 to 1.30.2 There have been a few refresh updates from the July release (1.26.1) to the November release (1.30.2) of VS Code. Highlights are as follows: New Settings editor UI making it easy to modify Azure Data Studio settings. Multiline search improvements. Better macOS support. SQL Server 2019 Preview extension The Microsoft team has been moving features from the SQL Server 2019 preview extension into the core Azure Data Studio tool for decades now. Here is a summary of the features moved into the core tool: Jupyter Notebook support has been moved to Azure Data Studio. Bug fixes in the External Data wizards: New schemas typed into the table mapping controls were getting lost. This is fixed now. Oracle type mappings have been updated. For more information, check out the official March release notes for Azure Data Studio. Microsoft Azure reportedly chooses Xilinx chips over Intel Altera for AI co-processors, says Bloomberg report Microsoft Azure now supports NVIDIA GPU Cloud (NGC) Microsoft Azure’s new governance DApp: An enterprise blockchain without mining

Spotify announced, last week, that it has come out with Chartify, a new open source Python data visualization library, making it easy for data scientists to create charts. It comes with features such as concise and user-friendly syntax and consistent data formatting among others. Let’s have a look at these features in this new library. Concise and user-friendly syntax Despite the abundance of tools such as Seaborn, Matplotlib, Plotly, Bokeh, etc, used by data scientists at Spotify, chart creation has always been a major issue in the data science workflow. Chartify solves that problem as the syntax in it is considerably more concise and user-friendly, as compared to the other tools. There are suggestions added in the docstrings, allowing users to recall the most common formatting options. This, in turn, saves time, allowing data scientists to spend less time on configuring chart aesthetics, and more on actually creating charts. Consistent data formatting Another common problem faced by data scientists is that different plotting methods need different input data formats, requiring users to completely reformat their input data. This leads to data scientists spending a lot of time manipulating data frames into the right state for their charts. Chartify’s consistent input data formatting allows you to quickly create and iterate on charts since less time is spent on data munging. Chartify Other features Since a majority of the problems could be solved by just a few chart types, Chartify focuses mainly on these use cases and comes with a complete example notebook that presents the full list of chart types that Chartify is capable of generating. Moreover, adding color into charts greatly help simplify the charting process, which is why Chartify has different palette types aligned to the different use cases for color. Additionally, Chartify offers support for Bokeh, an interactive python library for data visualization, providing users the option to fall back on manipulating Chartify charts with Bokeh if they need more control. For more information, check out the official Chartify blog post. cstar: Spotify’s Cassandra orchestration tool is now open source! Spotify has “one of the most intricate uses of JavaScript in the world,” says former engineer 8 ways to improve your data visualizations

[box type="shadow" align="" class="" width=""]Dr. Brandon: Hello and welcome to another episode of 'Date with Data Science'. Today we are going to talk about a topic that is all the rage these days in the data science community: Transfer Learning.  Jon: 'Transfer learning' sounds all sci-fi to me. Is it like the thing that Prof. X does in X-men reading other people's minds using that dome-like headset thing in his chamber? Dr. Brandon: If we are going to get X-men involved, what Prof. X does is closer to deep learning. We will talk about that another time. Transfer learning is simpler to explain. It's what you actually do everytime you get into some character, Jon.  Say, you are given the role of  Jack Sparrow to play. You will probably read a lot about pirates, watch a lot of pirate movies and even Jonny Depp in character and form your own version of Jack Sparrow. Now after that acting assignment is over, say you are given the opportunity to audition for the role of Captain Hook, the famous pirate from Peter Pan. You won't do your research from ground zero this time. You will retain general mannerisms of a Pirate you learned from your previous role, but will only learn the nuances of Captain Hook, like acting one-handed. Jon: That's pretty cool! So you say machines can also learn this way? Dr.Brandon: Of course, that's what transfer learning is all about: learn something, abstract the learning sufficiently, then apply it to another related problem. The following is an excerpt from a book by Kuntal Ganguly titled Learning Generative Adversarial Networks.[/box] Pre-trained models are not optimized for tackling user specific datasets, but they are extremely useful for the task at hand that has similarity with the trained model task. For example, a popular model, InceptionV3, is optimized for classifying images on a broad set of 1000 categories, but our domain might be to classify some dog breeds. A well-known technique used in deep learning that adapts an existing trained model for a similar task to the task at hand is known as Transfer Learning. And this is why Transfer Learning has gained a lot of popularity among deep learning practitioners and in recent years has become the go-to technique in many real-life use cases. It is all about transferring knowledge (or features) among related domain. Purpose of Transfer Learning Let say you have trained a deep neural network to differentiate between fresh mango and rotten mango. During training, the network requires thousands of rotten and fresh mango images and hours of training to learn knowledge like if any fruit is rotten, a liquid will ooze out of the fruit and it produce a bad odor. Now with this training experience the network, can be used for different task/use-case to differentiate between a rotten apple and fresh apple using the knowledge of rotten features learned during training of mango images. The general approach of Transfer Learning is to train a base network and then copy its first n layers to the first n layers of a target network. The remaining layers of the target network are initialized randomly and trained toward the targeted use-case. The main scenarios for using Transfer Learning in your deep learning workflow are as follows: Smaller datasets: When you have a smaller dataset, building a deep learning model from scratch won't work well. Transfer Learning provides the way to apply a pre-trained model to new classes of data. Let's say a pre-trained model built from one million images of ImageNet data will converge to a decent solution (after training on just a fraction of the available smaller training data, for example, CIFAR-10) compared to a deep learning model built with a smaller dataset from scratch. Less resource: Deep learning process (such as convolution) requires a significant amount of resource and time. Deep learning process are well suited to run on high graded GPU-based machines. But with pre-trained models, you can easily train across a full training set (let's say 50000 images) in less than a minute using your laptop/notebook without GPU, since the majority of time a model is modified in the final layer with a simple update of just a classifier or regressor. Various approaches of using pre-trained models Using pre-trained architecture: Instead of transferring weights of the trained model, we can only use the architecture and initialize our own random weights to our new dataset. Feature extractor: A pre-trained model can be used as a feature extraction mechanism just by simply removing the output layer of the network (that gives the probabilities for being in each of the n classes) and then freezing all the previous layers of the network as a fixed feature extractor for the new dataset. Partially freezing the network: Instead of replacing only the final layer and extracting features from all previous layers, sometime we might train our new model partially (that is, to keep the weights of initial layers of the network frozen while retraining only the higher layers). Choice of the number of frozen layers can be considered as one more hyper-parameter. Next, read about how transfer learning is being used in the real world. If you enjoyed the above excerpt, do check out the book it is from.

Neal Myrick Global Head of the Tableau Foundation Kristin Adderson December 28, 2020 - 9:59pm December 27, 2020 Addressing a global pandemic and economic crisis while also driving for change in other areas—from racial inequity to equitable education to hunger—is a monumental challenge. We are lucky to have amazing nonprofit partners tackling these issues. We took a moment to check-in with some to hear how the year has shaped—and reshaped—their approach to solving some of the world’s most pressing issues.  Driving for racial equity and justice amid the pandemic: PolicyLink “2020 was tragic and heart-opening for racial equity,” says Josh Kirschenbaum, Chief Operating Officer of the racial equity research and action institute PolicyLink. COVID-19 exposed racial disparities in health and access to care, and the murder of George Floyd and the protests that followed showed how far the country has to go to address them. “We have to step into this opening and move into an era of reckoning and acceleration around equity and justice that we’ve never seen before,” he says. Over the past year, PolicyLink helped drive the conversation around the need for equity-based solutions to COVID-19 with a comprehensive plan and set of policy priorities for pandemic response. They also released a weekly publication called COVID, Race, and the Revolution. “It’s critical to connect our data and policy proposals with narrative and communications,” Kirschenbaum says.  PolicyLink has also worked to draw attention to the broader racial inequity crisis in the U.S. This summer, they released their Racial Equity Index, a data tool to measure the state of equity in the largest 100 metros across the U.S. They also released a report outlining racial disparities in the workforce during the pandemic and were a founding partner in WeMustCount.org, an effort to push for COVID-19 data disaggregated by race. In 2021, PolicyLink wants to transform the energy and data around racial disparities in the U.S. into structural change. “We are no longer at the level of just doing project-based work,” Kirschenbaum says. “This is the time to lead with transformative solidary, focus on equity, and really redesign the nation.” Combatting increasing hunger: Feeding America and World Food Programme Image credit: Feeding AmericaCOVID-19 is a multi-level crisis. Our partners at Feeding America and the World Food Programme have seen firsthand how the pandemic has affected hunger in the U.S. and the world—and they’re working to respond to it.  “There’s been a perfect storm of increased demand, declines in donations of food, and disruptions to the charitable food assistance system’s operating model,” says Christine Feiner, Feeding America’s director of corporate partnerships. The organization estimates that progress made against food insecurity in the U.S.—which before was at the lowest it had been in 20 years—will be wiped out due to COVID-19. Over the last year, Feeding America saw demand increase 60% across its network of 200 food banks. Feeding America has relied on data to guide the organization through the pandemic. They launched a survey to understand demand and challenges across their member food banks. “That allowed us to have a real-time view into what food banks were seeing on the ground so we could property support them and connect them to additional resources,” Feiner says. Feeding America has also used this data to push for policy change at the federal level to help people at risk of hunger—work they plan to continue next year.  The United Nations World Food Programme—which became a Nobel Peace Prize Laureate this year—has been contending with increased need globally. “Roughly a quarter of a billion people—especially the already poor—are expected to have experienced food insecurity this year, largely driven by the loss of jobs, remittances, and purchasing power. Already poor and food insecure populations are disproportionately affected,” says Pierre Guillaume Wielezynski, the digital transformation services chief at WFP.  With the pandemic limiting WFP’s ability to work directly in communities and deliver aid, they’ve been able to use data and technology to reach people in need. In Tableau, they built a shipping and logistics platform for the entire humanitarian sector to manage and track aid deliveries in real-time. And they’ve been able to analyze data from technologies like text messaging and chatbots to get a picture of needs on the ground and ensure they’re responding most helpfully and efficiently.  Next year, WFP will continue to focus on delivering aid in communities while pushing for policy change, Wielezynski says. “Our presence in over 80 countries gives us a unique position to help advise our government partners on solutions to hunger and food insecurity,” he says.  Keeping the spotlight on homelessness: Community Solutions Image credit: Community SolutionsSince the beginning of the pandemic, the spotlight has been on frontline workers: healthcare professionals, post office workers, grocery store clerks. “What a lot of people didn’t really recognize initially was that homeless service providers are also on the frontline of protecting a population that is especially vulnerable to COVID-19,” says Anna Kim, communications lead for Community Solutions.  Communities and agencies that work with Community Solutions through their Built for Zero initiative—a data-driven program to bring about a functional end to homelessness—had to expand from the already-steep task of doing homeless response work to emergency pandemic response. “They needed to figure out how to get masks and PPE, and how to make shelters safe,” Kim says. But communities that have already been collecting detailed, person-specific data on their homeless population through Built for Zero found that same data to be critical in responding to COVID-19. Communities like Jacksonville, Florida, were able to use their by-name list of people experiencing homelessness to conduct wide-spread testing and keep people safe.  Throughout the pandemic, Community Solutions has elevated the importance of addressing homelessness as both a public health and racial equity imperative. “The raised public consciousness around racial equity after the murder of George Floyd has also heightened the importance of understanding how homelessness has always disproportionately impacted Black and Native populations,” Kim says. “We’ve been able to raise awareness of the need to invest even further in addressing these disparities and ending homelessness.” Community Solutions was recently named a finalist in the prestigious MacArthur Foundation 100&Change competition for their exceptional work. Next year, they hope to expand partnerships with cities across the U.S. to continue driving for an end to homelessness—even in the face of enormous health and economic challenges. Addressing growing education equity gaps: Equal Opportunity Schools As COVID-19 has forced schools to close and learning to go remote, equity divides among students have grown even more pronounced. “We talk about how COVID-19 is exacerbating inequities and pre-existing conditions in health, but it’s also true in education,” says Sasha Rabkin, chief strategy officer for Equal Opportunity Schools, an organization focused on closing equity gaps in education. “And inequity is a pre-existing condition.” EOS has built data tools for schools and districts to understand inequities and how they play out along racial lines. Through the surveys they conducted twice in 2020, EOS found that over 75% of students say that they are struggling with motivation–particularly with balancing coursework with the desire to have deep conversations about what’s happening globally with COVID, racial injustice, and political movements. “For educators to be able to hear that is invaluable,” Rabkin says. What’s on the mind of EOS and the educators they work with is how they can more genuinely meet students where they are and construct learning environments that respond to the current moment and bring students along. “Can we start to think about measuring and understanding and engaging with what matters, instead of continuing with the status quo? Schools look a lot like they did 20 years ago. Can we make this a moment to think critically about what we could be doing differently?” Supporting access to sanitation and hand-washing infrastructure: Splash A Splash handwashing station (Image credit: Make Beautiful)As a nonprofit, Splash focuses on providing handwashing, hygiene, and sanitation infrastructure to kids in schools and orphanages in cities throughout the Global South. During the pandemic, says Laura Mapp, Director of Business Development at Splash, their work has become even more essential and complicated. “At the beginning of the pandemic, we engaged in direct COVID relief with our government partners in Ethiopia,” Mapp says. In Addis Ababa, three of the schools where Splash had previously installed hand-washing stations and sanitation infrastructure became quarantine centers, where people who suspected they had the virus could safely quarantine away from their families. Splash also partnered with the Bureau of Health in Addis Ababa to bring their handwashing stations to six hospitals across the city. They’ve been able to install sanitation infrastructure in schools while children are learning remotely. Students learning from home, Mapp says, spurred Splash to innovate on ways to reach them virtually with messaging about the importance of handwashing and information about menstrual health, especially for girls. “This is helping us forge some new partnerships to enable the delivery of these tools, particularly in India, where mobile and computer usage is more accessible,” Mapp says. For instance, they’re partnering with a platform called Oky, designed by Unicef, that young girls can use to get answers about menstrual health questions. While the pandemic continues to pose significant challenges in the communities where Splash works, Mapp is hopeful that the increased attention on the need for good sanitation infrastructure and communication around hygiene best practices will help keep people safe through and beyond the pandemic. Pivoting a successful social enterprise to meet community needs: FareStart Image Credit: FareStartAs soon as COVID-19 began forcing lockdowns in cities across the U.S., FareStart knew it would have to pivot its operations. The social enterprise manages a handful of restaurants and cafes across Seattle, where people facing barriers to employment—from homelessness to drug-use history—gain training and experience in the workforce. With restaurants shuttering and in-person work discouraged, FareStart’s programs could not continue as normal.  Almost immediately, FareStart started using its restaurants and kitchens to prepare individual meals to deliver to the most vulnerable. FareStart now is serving over 50,000 individual meals per week, distributed across over 100 sites, says Erika Van Merr, FareStart’s Associate Director of Philanthropy.  Managing this broad distribution operation and network, Van Merr says, has required more data than FareStart ever used before. They’ve been using external data to understand the COVID-19 situation in their community while inputting data daily to track meals: preparation location, for what organization, and where they were distributed. “We really had to up our data savviness to make decisions about how to operate daily,” Van Merr says. They plan to continue using data to expand their community meals program even after the pandemic is over. While the organization has launched virtual training programs, it looks forward to bringing back the students in person and reopening its restaurant and cafes. “When people ask what our plans look like for next year, I tell them that we will continue to provide hunger relief for our community’s most vulnerable neighbors,” says Van Merr.  To learn more about Tableau Foundation and its partners, visit tableau.com/foundation.

Social media platforms have come under considerable criticism for allowing controversial media outlets for as long as 'fake news' has been in the public lexicon. But over the past week, major actions against Alex Jones' content channels suggests that things might be changing. Apple has pulled 5 out of Jones 6 podcasts from iTunes (first reported by Buzzfeed News), while hours later on Monday 6 August Facebook announced it was removing four of Jones' pages for breaching the platform's content guidelines. Alongside Facebook's and Apple's actions, Spotify also made the decision to remove Jones' content from the streaming platform and revoke his ability to publish "due to repeated violations of Spotify's prohibited content policies" according to a Spotify spokesperson. This news comes just weeks after YouTube removed a number of Infowars videos over 'hate speech' and initiated a 90 day ban on Infowars broadcasting live via YouTube. Unsurprisingly, the move has come under attack from those who see the move as an example of censorship. Even people critical of Jones' politics have come out to voice their unease: https://twitter.com/realJoeBarnes/status/1026466888744947721 However, elsewhere, the move is viewed positively with commentators suggesting social media platforms are starting to take responsibility for the content published on their systems. https://twitter.com/shannoncoulter/status/1025401502033039362 One thing that can be agreed is that the situation is a little confusing at the moment. And although it's true that it's time for Facebook, and other platforms to take more responsibility for what they publish, there are still issues around governance and consistency that need to be worked through and resolved. Facebook's action against Alex Jones - a recent timeline On July 27, Alex Jones was hit with a 30 day suspension by Facebook after the company removed 4 videos from its site that contravened its content guidelines. However, as numerous outlets reported at the time, this ban only effected Jones personally. His channels (like The Alex Jones Channel and Infowars) weren't impacted. However, those pages that weren't hit by Jones' personal ban have now been removed by Facebook. In a post published August 6, Facebook explained: "...we removed four videos on four Facebook Pages for violating our hate speech and bullying policies. These pages were the Alex Jones Channel Page, the Alex Jones Page, the InfoWars Page and the Infowars Nightly News Page..." The post also asserts that the ban is about violation of community standards not 'false news'. "While much of the discussion around Infowars has been related to false news, which is a serious issue that we are working to address by demoting links marked wrong by fact checkers and suggesting additional content, none of the violations that spurred today’s removals were related to this." Apple's action against Alex Jones Apple's decision to remove 5 of Alex Jones podcasts is, according to Buzzfeed News, "one of the largest enforcement actions intended to curb conspiratorial news content by a technology company to date." Like Facebook, Apple's decision was based on the content's "hate speech" rather than anything to do with 'fake news'. An Apple spokesperson explained to Buzzfeed News: "Apple does not tolerate hate speech, and we have clear guidelines that creators and developers must follow to ensure we provide a safe environment for all of our users... Podcasts that violate these guidelines are removed from our directory making them no longer searchable or available for download or streaming. We believe in representing a wide range of views, so long as people are respectful to those with differing opinions.” Spotify's action against Alex Jones' podcasts Spotify removed all episodes of The Alex Jones Show podcast on Monday 6 August. This follows the music streaming platform pulling a number of individual episodes of Jones' podcast at the beginning of August. This appears to be a consequence of Spotify's new content guidelines, updated in May 2018, which prohibits "hate content." The takeaway: there's still considerable confusion over content What this debacle shows is that there's confusion about how social media platforms should deal with content that it effectively publishes. Clearly, the likes of Facebook are trying to walk a tightrope that's going to take some time to resolve. The broader question is not just do we want to police the platforms billions of people use, but how we do that as well. Arguably, social media is at the center of today's political struggles, with many of them unsure how to manage the levels of responsibility that have landed on on their algorithms. Read next Time for Facebook, Twitter and other social media to take responsibility or face regulation Why Wall Street unfriended Facebook: Stocks fell $120 billion in market value after Q2 2018 earnings call Spotify has “one of the most intricate uses of JavaScript in the world,” says former engineer  

After releasing PostgreSQL 11 beta 1, back in May, the PostgreSQL Global Development Group finally released PostgreSQL 11, yesterday. PostgreSQL 11 explores features such as increased performance for partitioning, support for transactions in stored procedures, improved capabilities for query parallelism, and Just-in-Time (JIT) compilation for expressions among other updates. PostgreSQL is a popular open source relational database management system that offers better reliability, robustness, and enhanced performance measures. Let’s have a look at these features in PostgreSQL 11. Increased performance for partitioning PostgreSQL 11 comes with an ability to partition the data using a hash key, which is known as hash partitioning. This adds to the already existing ability to partition data in PostgreSQL using a list of values or by a range. Moreover, PostgreSQL 11 also improves the data federation abilities by implementing functionality improvements for partitions using PostgreSQL foreign data wrapper, and postgres_fdw. For managing these partitions, PostgreSQL 11 comes with a “catch-all” default partition for data that doesn’t match a partition key. It also comes with an ability to create primary keys, foreign keys, indexes as well as triggers on partitioned tables. The latest release also offers support for automatic movement of rows to the correct partition, given that the partition key for that row is updated. Additionally, PostgreSQL 11 enhances the query performance when reading from partitions with the help of a new partition elimination strategy. It also offers support for the popular "upsert" feature on partitioned tables. The upsert feature helps users simplify the application code as well as reduce the network overhead when interacting with their data. Support for transactions in stored procedures With PostgreSQL 11 comes newly added SQL procedures that help perform full transaction management within the body of a function. This enables the developers to build advanced server-side applications like the ones that involve incremental bulk data loading. Also, SQL procedures can now be created using the CREATE PROCEDURE command which is executed using the CALL command. These SQL procedures are supported by the server-side procedural languages such as PL/pgSQL, PL/Perl, PL/Python, and PL/Tcl. Improved capabilities for query parallelism PostgreSQL 11 enhances the parallel query performance, using the performance gains in parallel sequential scans and hash joins. It also performs more efficient scans of the partitioned data. PostgreSQL 11 comes with added parallelism for a range of data definitions commands, especially for the creation of B-tree indexes generated by executing the standard CREATE INDEX command. Other data definition commands that either create tables or materialize the views from queries are also enabled with parallelism. This includes the CREATE TABLE .. AS, SELECT INTO, and CREATE MATERIALIZED VIEW. Just-in-Time (JIT) compilation for expressions PostgreSQL 11 offers support for Just-In-Time (JIT) compilation, This helps to accelerate the execution of certain expressions during query execution. The JIT expression compilation uses the LLVM project to boost the execution of expressions in WHERE clauses, target lists, aggregates, projections, as well as some other internal operations. Other Improvements ALTER TABLE .. ADD COLUMN .. DEFAULT ..have been replaced with a not NULL default to rewrite the whole table on execution. This offers a significant performance boost when running this command. Additional functionality has been added for working with window functions, including allowing RANGE to use PRECEDING/FOLLOWING, GROUPS, and frame exclusion. Keywords such as "quit" and "exit" have been added to the PostgreSQL command-line interface to help make it easier to leave the command-line tool. For more information, check out the official release notes. PostgreSQL group releases an update to 9.6.10, 9.5.14, 9.4.19, 9.3.24 How to perform data partitioning in PostgreSQL 10 How to write effective Stored Procedures in PostgreSQL

Yesterday, the ninth circuit court of appeals gave a big win for web accessibility in a case against Domino’s Pizza. In 2016, a blind man filed a federal lawsuit against Domino’s stating that it wasn’t compatible with standard screen reading software which is designed to vocalize text and visual information. This did not allow him to use the pizza builder feature to personalize his order. Per his claim, Domino’s violated the Americans With Disabilities Act and should make its online presence compatible with Web Content Accessibility Guidelines. A blog post published by the National Retail Federation highlights that such lawsuits are on the rise, with 1,053 filed in the first half of last year compared to 814 in all of 2017. All of them voiced how there is a lack of clarity in how the ADA applies to the modern internet. [box type="shadow" align="" class="" width=""]Web Content Accessibility Guidelines (WCAG) is developed through the W3C process with the goal of providing a single shared standard for web content accessibility. The WCAG documents explain how to make web content more accessible to people with disabilities.[/box] Earlier, a lower court ruled in favor of Domino’s and tossed the case out of court. However, the court of appeals court reversed the ruling saying that the ADA covers websites and mobile applications and so the case is relevant. Domino’s argued that there was an absence of regulations specifically requiring web accessibility or referencing the Web Content Accessibility Guidelines. However the appellate judges explained the case was not about whether Domino’s did not comply with WCAG  “While we understand why Domino’s wants DOJ to issue specific guidelines for website and app accessibility, the Constitution only requires that Domino’s receive fair notice of its legal duties, not a blueprint for compliance with its statutory obligations,” U.S. Circuit Judges John B. Owens wrote in a 25-page opinion. The judges' panel said the case was relevant and send it back to the district court. They will talk about whether the Domino’s website and app comply with the ADA mandate to “provide the blind with effective communication and full and equal enjoyment of its products and services.” A Twitter thread by Jared Spool, applauded the court’s actions to apply web accessibility to ADA penalties and talked about the long and short term implications of this news. The first will likely come when insurance companies raise rates for any company that doesn’t meet WCAG compliance. This will create a bigger market for compliance certification firms. Insurance companies will demand certification sign-off to give preferred premiums. This will likely push companies to require WCAG understanding from the designers they hire. In the short term, we’ll likely see a higher demand for UX professionals with specialized knowledge in accessibility. In the long term, this knowledge will be required for all UX professionals. The demand for specialists will likely decrease as it becomes common practice. Also, toolkits, frameworks, and other standard platforms will build accessibility in. This will also reduce the demand for specialists, as it will become more difficult to build things that aren’t accessible. Good, accessible design will become the path of least resistance. You may go through the full appeal from the United States District Court for the Central District of California. EFF asks California Supreme Court to hear a case on government data accessibility and anonymization under CPRA 7 Web design trends and predictions for 2019 We discuss the key trends for web and app developers in 2019 [Podcast]

Advances in machine learning have greatly improved products, processes, and research, and how people might interact with computers. One of the factors lacking in machine learning processes is the ability to give an explanation for their predictions. The inability to give a proper explanation of results leads to end-users losing their trust over the system, which ultimately acts as a barrier to the adoption of machine learning. Hence, along with the impressive results from machine learning, it is also important to understand why and where it works, and when it won’t. In this article, we will talk about some ways to increase machine learning interpretability and make predictions from machine learning models understandable. 3 interesting methods for interpreting Machine Learning predictions According to Miller, interpretability is the degree to which a human can understand the cause of a decision. Interpretable predictions lead to better trust and provide insight into how the model may be improved. The kind of machine learning developments happening in the present times require a lot of complex models, which lack in interpretability. Simpler models (e.g. linear models), on the other hand,  often give a correct interpretation of a prediction model’s output, but they are often less accurate than complex models. Thus creating a tension between accuracy and interpretability. Complex models are less interpretable as their relationships are generally not concisely summarized. However, if we focus on a prediction made on a particular sample, we can describe the relationships more easily. Balancing the trade-off between model complexity and interpretability lies at the heart of the research done in the area of developing interpretable deep learning and machine learning models. We will discuss a few methods to increase the interpretability of complex ML models by summarizing model behavior with respect to a single prediction. LIME or Local Interpretable Model-Agnostic Explanations, is a method developed in the paper Why should I trust you? for interpreting individual model predictions based on locally approximating the model around a given prediction. LIME uses two approaches to explain specific predictions: perturbation and linear approximation. With Perturbation, LIME takes a prediction that requires explanation and systematically perturbs its inputs. These perturbed inputs become new, labeled training data for a simpler approximate model. It then does local linear approximation by fitting a linear model to describe the relationships between the (perturbed) inputs and outputs. Thus a simple linear algorithm approximates the more complex, nonlinear function. DeepLIFT (Deep Learning Important FeaTures) is another method which serves as a recursive prediction explanation method for deep learning.  This method decomposes the output prediction of a neural network on a specific input by backpropagating the contributions of all neurons in the network to every feature of the input. DeepLIFT assigns contribution scores based on the difference between activation of each neuron and its ‘reference activation’. DeepLIFT can also reveal dependencies which are missed by other approaches by optionally giving separate consideration to positive and negative contributions. Layer-wise relevance propagation is another method for interpreting the predictions of deep learning models. It determines which features in a particular input vector contribute most strongly to a neural network’s output.  It defines a set of constraints to derive a number of different relevance propagation functions. Thus we saw 3 different ways of summarizing model behavior with a single prediction to increase model interpretability. Another important avenue to interpret machine learning models is to understand (and rethink) generalization. What is generalization and how it affects Machine learning interpretability Machine learning algorithms are trained on certain datasets, called training sets. During training, a model learns intrinsic patterns in data and updates its internal parameters to better understand the data. Once training is over, the model is tried upon test data to predict results based on what it has learned. In an ideal scenario, the model would always accurately predict the results for the test data. In reality, what happens is that the model is able to identify all the relevant information in the training data, but sometimes fails when presented with the new data. This difference between “training error” and “test error” is called the generalization error. The ultimate aim of turning a machine learning system to a scalable product is generalization. Every task in ML wants to create a generalized algorithm that acts in the same way for all kind of distributions. And the ability to distinguish models that generalize well from those that do not, will not only help to make ML models more interpretable, but it might also lead to more principled and reliable model architecture design. According to the conventional statistical theory, small generalization error is either due to properties of the model family or because of the regularization techniques used during training. A recent paper at ICLR 2017,  Understanding deep learning requires rethinking generalization shows that current machine learning theoretical frameworks fail to explain the impressive results of deep learning approaches and why understanding deep learning requires rethinking generalization. They support their findings through extensive systematic experiments. Developing human understanding through visualizing ML models Interpretability also means creating models that support human understanding of machine learning. Human interpretation is enhanced when visual and interactive diagrams and figures are used for the purpose of explaining the results of ML models. This is why a tight interplay of UX design with Machine learning is essential for increasing Machine learning interpretability. Walking along the lines of Human-centered Machine Learning, researchers at Google, OpenAI, DeepMind, YC Research and others have come up with Distill. This open science journal features articles which have a clear exposition of machine learning concepts using excellent interactive visualization tools. Most of these articles are aimed at understanding the inner working of various machine learning techniques. Some of these include: An article on attention and Augmented Recurrent Neural Networks which has a beautiful visualization of attention distribution in RNN. Another one on feature visualization, which talks about how neural networks build up their understanding of images Google has also launched the PAIR initiative to study and design the most effective ways for people to interact with AI systems. It helps researchers understand ML systems through work on interpretability and expanding the community of developers. R2D3 is another website, which provides an excellent visual introduction to machine learning. Facets is another tool for visualizing and understanding training datasets to provide a human-centered approach to ML engineering. Conclusion Human-Centered Machine Learning is all about increasing machine learning interpretability of ML systems and in developing their human understanding. It is about ML and AI systems understanding how humans reason, communicate and collaborate. As algorithms are used to make decisions in more angles of everyday life, it’s important for data scientists to train them thoughtfully to ensure the models make decisions for the right reasons. As more progress is done in this area, ML systems will not make commonsense errors or violate user expectations or place themselves in situations that can lead to conflict and harm, making such systems safer to use.  As research continues in this area, machines will soon be able to completely explain their decisions and their results in the most humane way possible.