Tech Guides - Data

The paper, Parameter space noise for exploration proposes parameter space noise as an efficient solution for exploration, a big problem for deep reinforcement learning. This paper is authored by Pieter Abbeel, Matthias Plappert, Rein Houthooft, Prafulla Dhariwal, Szymon Sidor, Richard Y. Chen, Xi Chen, Tamim Asfour, and Marcin Andrychowicz. Pieter Abbeel is currently a professor at UC Berkeley since 2008. He was also a Research Scientist at OpenAI (2016-2017). Pieter is one of the pioneers of deep reinforcement learning for robotics, including learning locomotion and visuomotor skills. His current research focuses on robotics and machine learning with particular focus on deep reinforcement learning, meta-learning, and AI safety. Deep reinforcement learning is the combination of deep learning with reinforcement learning to create artificial agents to achieve human-level performance across many challenging domains. This article will talk about one of Pieter’s top accepted research papers in the field of deep reinforcement learning at the 6th annual ICLR conference scheduled to happen between April 30 - May 03, 2018. Improving the exploratory behavior of Deep RL algorithms with Parameter Space Noise What problem is the paper attempting to solve? This paper is about the exploration challenge in deep reinforcement learning (RL) algorithms. The main purpose of exploration is to ensure that the agent’s behavior does not converge prematurely to a local optimum. Enabling efficient and effective exploration is difficult since it is not directed by the reward function of the underlying Markov decision process (MDP). A large number of methods have been proposed to tackle this challenge in high-dimensional and/or continuous-action MDPs. These methods increase the exploratory nature of these algorithms through the addition of temporally-correlated noise or through the addition of parameter noise. The main limitation of these methods is that they are either only proposed and evaluated for the on-policy setting with relatively small and shallow function approximators or disregard all temporal structure and gradient information. Paper summary This paper proposes adding noise to the parameters (parameter space noise) of a deep network when taking actions in deep reinforcement learning to encourage exploration. The effectiveness of this approach is demonstrated through empirical analysis across a variety of reinforcement learning tasks (i.e.DQN, DDPG, and TRPO). It answers the following questions: Do existing state-of-the-art RL algorithms benefit from incorporating parameter space noise? Does parameter space noise aid in exploring sparse reward environments more effectively? How does parameter space noise exploration compare against evolution strategies for deep policies with respect to sample efficiency? Key Takeaways The paper describes a method which proves parameter space noise as a conceptually simple yet effective replacement for traditional action space noise like -greedy and additive Gaussian noise. This work shows that parameter perturbations can successfully be combined with contemporary on- and off-policy deep RL algorithms such as DQN, DDPG, and TRPO and often results in improved performance compared to action noise. The paper attempts to prove with experiments that using parameter noise allows solving environments with very sparse rewards, in which action noise is unlikely to succeed. Parameter space noise is a viable and interesting alternative to action space noise, which is still the effective standard in most reinforcement learning applications. Reviewer feedback summary Overall Score: 20/30 Average Score: 6.66 The reviewers were pleased with the paper. They termed it as a simple strategy for exploration that is effective empirically. The paper was found to be clear and well written with thorough experiments across deep RL domains.  The authors have also released open-source code along with their paper for reproducibility, which was appreciated by the reviewers. However, a common trend among the reviews was that the authors overstated their claims and contributions.  The reviewers called out some statements in particular (e.g. the discussion of ES and RL). They also felt that the paper lacked a strong justification for the method other than it being empirically effective and intuitive.

The new phenomenon to hit the world of ‘Big Data’ seems to be ‘Deep Learning’. I’ve read many articles and papers where people question whether there’s a future for it, or if it’s just a buzzword that will die out like many a term before it. Likewise I have seen people who are genuinely excited and truly believe it is the future of Artificial intelligence; the one solution that can greatly improve the accuracy of our data and development of systems. Deep learning is currently a very active research area, by no means is it established as an industry standard, but rather one which is picking up pace and brings a strong promise of being a game changer when dealing with raw, unstructured data. So what is Deep Learning? Deep learning is a concept conceived from machine learning. In very simple terms, we think of machine learning as a method of teaching machines (using complex algorithms to form neural networks) to make improved predictions of outcomes based on patterns and behaviour from initial data sets.   The concept goes a step further however. The idea is based around a set of techniques used to train machines (Neural Networks) in processing information that can generate levels of accuracy nearly equivalent to that of a human eye. Deep learning is currently one of the best providers of solutions regarding problems in image recognition, speech recognition, object recognition and natural language processing. There are a growing number of libraries that are available, in a wide range of different languages (Python, R, Java) and frameworks such as: Caffe,Theanodarch, H20, Deeplearning4j, DeepDist etc.   How does Deep Learning work? The central idea is around ‘Deep Neural Networks’. Deep Neural Networks take traditional neural networks (or artificial neural networks) and build them on top of one another to form layers that are represented in a hierarchy. Deep learning allows each layer in the hierarchy to learn more about the qualities of the initial data. To put this in perspective; the output of data in level one is then the input of data in level 2. The same process of filtering is used a number of times until the level of accuracy allows the machine to identify its goal as accurately as possible. It’s essentially a repeat process that keeps refining the initial dataset. Here is a simple example of Deep learning. Imagine a face, we as humans are very good at making sense of what our eyes show us, all the while doing it without even realising. We can easily make out ones: face shape, eyes, ears, nose, mouth etc. We take this for granted and don’t fully appreciate how difficult (and complex) it can get whilst writing programs for machines to do what comes naturally to us. The difficulty for machines in this case is pattern recognition - identifying edges, shapes, objects etc. The aim is to develop these ‘deep neural networks’ by increasing and improving the number of layers - training each network to learn more about the data to the point where (in our example) it’s equal to human accuracy. What is the future of Deep Learning? Deep learning seems to have a bright future for sure, not that it is a new concept, I would actually argue it’s now practical rather than theoretical. We can expect to see the development of new tools, libraries and platforms, even improvements on current technologies such as Hadoop to accommodate the growth of Deep Learning. However it may not be all smooth sailing. It is still by far very difficult and time consuming task to understand, especially when trying to optimise networks as datasets grow larger and larger, surely they will be prone to errors? Additionally, the hierarchy of networks formed would surely have to be scaled for larger complex and data intensive AI problems.     Nonetheless, the popularity around Deep learning has seen large organisations invest heavily, such as: Yahoo, Facebook, Googles acquisition of Deepmind for $400 million and Twitter’s purchase of Madbits. They are just few of the high profile investments amongst many. 2015 really does seem like the year Deep learning will show its true potential. Prepare for the advent of deep learning by ensuring you know all there is to know about machine learning with our article. Read 'How to do Machine Learning with Python' now. Discover more Machine Learning tutorials and content on our dedicated page. Find it here.

At the start of 2018, the cryptocurrency boom seemed to be at an end. Bitcoin's price plunged in just a matter of weeks from a mid-December 2017 high of $20,000 to less than $10,000. Suddenly everything seemed unpredictable and volatile. The cryptocurrency honeymoon was at an end. However, while many are starting to feel cautious about investing, a new cryptocurrency might change the game. BrickCoin might well be the cryptocurrency to reinvigorate a world that's shifted from optimism to pessimism in just a couple of months. But what is BrickCoin? And how is it different from other cryptocurrencies? Most importantly, why might you be confident in its success? What is BrickCoin? This one’s also a Blockchain based currency, but one backed with real estate(REITs). BrickCoin aims to be the first regulated, KYC, and AML compliant real estate crypto. The real estate is comprehensive to regulators and is accepted by many as a robust asset class. This is a major distinguishing point between BrickCoin and other cryptocurrencies. Traditional money saving methods - savings account and fixed deposits - are not inflation-proof. They also have very low levels of interest at the moment. On the other hand, complex investment options such as hedge funds are typically only available to wealthy individuals as they require large initial investments. These also do not offer ready liquidity and are vulnerable to bankruptcy. BrickCoin comes to the rescue here, as it claims to be Inflation proof. Find out more about BrickCoin here. The key features of BrickCoin It is a savings token which can be bought with traditional currency or digital currency. It represents an investment in a piece of commercial debt-free real estate. The real estate is held as part of a very secure, high-value, debt-free REIT. BrickCoins are kept in a mobile digital wallet. All transactions are fully-managed, validated and trackable by blockchain technology. BrickCoins can be converted into FIAT currency instantly. [box type="note" align="" class="" width=""]Also read about CryptoML, a machine learning powered cryptocurrency platform.[/box] BrickCoin is essentially the next step in the evolution of cryptocurrency. It is a savings scheme that is backed by a non-inflationary asset - commercial debt-free real estate - to deliver stable capital preservation. As a cryptocurrency, it allows savers to convert their money to and from BrickCoin tokens using the full security and convenience of blockchain technology. BrickCoin will be the first cryptocurrency that bridges the gap between the necessary reliance on the FIAT currencies and the asset-backed wealth-creation opportunities that are often out of reach for many ordinary savers. Crypto News Cryptojacking is a growing cybersecurity threat, report warns Coinbase Commerce API launches Crypto-ML, a machine learning powered cryptocurrency platform Crypto Op-Ed There and back again: Decrypting Bitcoin`s 2017 journey from $1000 to $20000 Will Ethereum eclipse Bitcoin? Beyond the Bitcoin: How cryptocurrency can make a difference in hurricane disaster relief Cryptocurrency Tutorials Predicting Bitcoin price from historical and live data How to mine bitcoin with your Raspberry Pi Protecting Your Bitcoins

Machine Learning is entangling the world in the web of smart, intelligent and automated algorithms. The next bug caught in the mesh is Networking. The paradigm of network management is all set to take a pretty big shift with machine learning sprinkled networks. Termed as Intent-based network, it is a network management system focussed on improving network availability and agility through means of automation.   Exploring an Intent-based Network Essentially, an Intent-based networking software (IBNS), like any other networking solution helps to plan, plot, execute, and operate networks. A distinguishable fact is, that the intent based network can translate the “intent” of a user. The user may be the end-user or the network administrator. The “intent” may be through a series of commands or through APIs. How does it work?— a network admin describes the state of a network, or the policy they want a network to follow, the IBNS validates whether such a request is feasible and then executes it. The second differentiating feature of an IBNS is the ability to perform automated implementation of the network state and policies described above. They can also control the security level to be applied to applications based on user role, time, and device conditions. Additionally, the vLANs, mandate firewalls and other network technologies required within the network can be automated in an IBNS.   An IBNS can monitor a network in real-time and take necessary actions to assess and improve the network. The ultimate goal of an IBNS is, to ensure that the desired condition of a network is maintained while making sure an automated reparative action is taken in case of any discrepancy. Why networks need machine learning to create intent? Traditional Networks cannot be integrated into networking environments after a certain number of layers. They require manual integration which can be a daunting process demanding advanced skill-set. Traditional network systems are also prone to failover and capacity augment as they involve managing individual devices. This resulted in developing a new set of centrally globalized network system, controlled from one place that permeates through the network. Machine learning was considered a possible contender for aiding the development of such networks and hence IBNS came to be. The idea of IBNS was proposed long ago, but requirements for developing such a system were not available.  In the recent times, advances in ML have made it possible to develop such a system which can automate tasks. By automation we mean, automating the entire network policy when the network administrator simply defines the desired state of the network.   With the presence of ML, IBNS can do real time analysis of network conditions, which  involves monitoring, identifying and reacting to real time data using algorithmic validations. Why IBNS could be the future of networking? Central control system The intent based network uses a central, cloud based control system to program the routes to be taken by the network elements to deliver required information. The control system can also reprogram them in case of a network path failure. Automation of network elements Intent Based networks are all about reducing human efforts. A network is built on millions of connected devices. Managing a huge number of devices will require a lot of manpower. Also, the current networking system will have difficulty in managing enormous amount of data exploding from IoT devices. With an IBNS, human intervention required to build these network elements across services and circuits is reduced by a considerable amount. IBNS provides an automated network management system which mathematically validates the desired state of networks as specified by network administrators. Real-time decision making The network systems continuously gather information about the network architecture while adapting themselves to the desired condition. If a network is congested at a particular time frame, it can modify the route. For example, the GPS system can re-route a driver if he/she encounters some sort of traffic jam or a closed lane.  IBNS can also gather information about the current status of the network by consistent ingestion of real-time state pointers in the networking system. The real time data, so gathered can in turn make it smarter and more predictive.   Business benefits gained Intent-based software will open better opportunities for IT employees.  Due to elimination of mundane operational tasks, IT admins can increase their productivity levels while improving their skill set. For an organization, it would mean lower cost of operation, reduced fallacy, better reliability & security, resource management, increased optimization, and multi-vendor device management. What do we see around the globe in the IBNS domain? The rising potential of IBNS is giving startups and MNCs higher opportunities to invest in this technology.   Apstra have launched their Apstra Operating system which provides deployment of an IBN system to a variety of customers including Cisco, Arista, Juniper, and other white box users. The solution focuses on a single data center. It prevents and repairs network outages to improve infrastructure agility. The AOS also provides network admins full transparency to intent with the ability to ask any question about the network, amend any aspect of the intent after deployment, while allowing complete freedom for users to perform customizations of the AOS. Anuta networks, the pioneer in Network service orchestration have announced their award winning NCX intent-based orchestration platform to automate network services, which has enabled DevOps for networking. They provide more than 35 industry leading vendors with orchestrated devices. They also own detailed and exhaustive REST API for network integration. One of the top pioneers in the networking domain, Cisco has launched itself in the intent-based networking space with the solution—Software Defined Access (SDA).  The software enables easy to perform operational tasks and build powerful networks by automating the network. It also boasts of features like: Multi-site management Improved visibility with simple topological views Integration with Kubernetes Zero -trust security With tech-innovation taking place all around, networking organizations are racing towards breaking the clutter by bringing their intent based networks to the networking space. According to Gartner analyst, Andrew Lerner, “Intent-based networking is nascent, but could be the next big thing in networking, as it promises to improve network availability and agility, which are key, as organizations transition to digital business. I&O leaders responsible for networking need to determine if and when to pilot this technology.”

Last November, Google open sourced its shiny Machine Intelligence package, promising a simpler way to develop deep learning algorithms that can be deployed anywhere, from your phone to a big cluster without a hassle. They even take advantage of running over GPUs for better performance. Let's Give It a Shot! First things first, let's install it: # Ubuntu/Linux 64-bit, CPU only (GPU enabled version requires more deps): $ sudo pip install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.7.1-cp27-none-linux_x86_64.whl # Mac OS X, CPU only:$ sudo easy_install --upgrade six$ sudo pip install --upgrade https://storage.googleapis.com/tensorflow/mac/tensorflow-0.7.1-cp27-none-any.whl We are going to play with the old-known iris dataset, where we will train a neural network to take dimensions of the sepals and petals of an iris plant and classify it between three different types of iris plants: Iris setosa, Iris versicolour, and Iris virginica. You can download the training CSV dataset from here. Reading the Training Data Because TensorFlow is prepared for cluster-sized data, it allows you to define an input by feeding it with a queue of filenames to process (think of MapReduce output shards). In our simple case, we are going to just hardcode the path to our only file: import tensorflow as tf def inputs(): filename_queue = tf.train.string_input_producer(["iris.data"]) We then need to set up the Reader, which will work with the file contents. In our case, it's a TextLineReader that will produce a tensor for each line of text in the dataset: reader = tf.TextLineReader()key, value = reader.read(filename_queue) Then we are going to parse each line into the feature tensor of each sample in the dataset, specifying the data types (in our case, they are all floats except the iris class, which is a string). # decode_csv will convert a Tensor from type string (the text line) in # a tuple of tensor columns with the specified defaults, which also # sets the data type for each column sepal_length, sepal_width, petal_length, petal_width, label = tf.decode_csv(value, record_defaults=[[0.0], [0.0], [0.0], [0.0], [""]]) # we could work with each column separately if we want; but here we # simply want to process a single feature vector containing all the # data for each sample. features = tf.pack([sepal_length, sepal_width, petal_length, petal_width]) Finally, in our data file, the samples are actually sorted by iris type. This would lead to bad performance of the model and make it inconvenient for splitting between training and evaluation sets, so we are going to shuffle the data before returning it by using a tensor queue designed for it. All the buffering parameters can be set to 1500 because that is the exact number of samples in the data, so will store it completely in memory. The batch size will also set the number of rows we pack in a single tensor for applying operations in parallel: return tf.train.shuffle_batch([features, label], batch_size=100, capacity=1500, min_after_dequeue=100)   Converting the Data Our label field on the training dataset is a string that holds the three possible values of the Iris class. To make it friendly with the neural network output, we need to convert this data to a three-column vector, one for each class, where the value should be 1 (100% probability) when the sample belongs to that class. This is a typical transformation you may need to do with input data. def string_label_as_probability_tensor(label): is_setosa = tf.equal(label, ["Iris-setosa"]) is_versicolor = tf.equal(label, ["Iris-versicolor"]) is_virginica = tf.equal(label, ["Iris-virginica"]) return tf.to_float(tf.pack([is_setosa, is_versicolor, is_virginica]))   The Inference Model (Where the Magic Happens) We are going to use a single neuron network with a Softmax activation function. The variables (learned parameters of our model) will only be the matrix weights applied to the different features for each sample of input data. # model: inferred_label = softmax(Wx + b) # where x is the features vector of each data example W = tf.Variable(tf.zeros([4, 3])) b = tf.Variable(tf.zeros([3])) def inference(features): # we need x as a single column matrix for the multiplication x = tf.reshape(features, [1, 4]) inferred_label = tf.nn.softmax(tf.matmul(x, W) + b) return inferred_label Notice that we left the model parameters as variables outside of the scope of the function. That is because we want to use those same variables both while training and when evaluating and using the model. Training the Model We train the model using backpropagation, trying to minimize cross entropy, which is the usual way to train a Softmax network. At a high level, this means that for each data sample, we compare the output of the inference with the real value and calculate the error (how far we are). Then we use the error value to adjust the learning parameters in a way that minimizes that error. We also have to set the learning factor; it means for each sample, how much of the computed error we will apply to correct the parameters. There has to be a balance between the learning factor, the number of learning loop cycles, and the number of samples we pack tighter in the same tensor in batch; the bigger the batch, the smaller the factor and the higher the number of cycles. def train(features, tensor_label): inferred_label = inference(features) cross_entropy = -tf.reduce_sum(tensor_label*tf.log(inferred_label)) train_step = tf.train.GradientDescentOptimizer(0.001) .minimize(cross_entropy) return train_step Evaluating the Model We are going to evaluate our model using accuracy, which is the ratio of cases where our network identifies the right iris class over the total evaluation samples. def evaluate(evaluation_features, evaluation_labels): inferred_label = inference(evaluation_features) correct_prediction = tf.equal(tf.argmax(inferred_label, 1), tf.argmax(evaluation_labels, 1)) accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32)) return accuracy Running the Model We are only left to connect our graph and run it in a session, where the defined operations are actually going to use the data. We also split our input data between training and evaluation around 70%:30%, and run a training loop with it 1,000 times. features, label = inputs() tensor_label = string_label_as_probability_tensor(label) train_step = train(features[0:69, 0:4], tensor_label[0:69, 0:3]) evaluate_step = evaluate(features[70:99, 0:4], tensor_label[70:99, 0:3]) with tf.Session() as sess: sess.run(tf.initialize_all_variables()) # Start populating the filename queue. coord = tf.train.Coordinator() threads = tf.train.start_queue_runners(coord=coord) for i in range(1000): sess.run(train_step) print sess.run(evaluate_step) # should print 0 => setosa print sess.run(tf.argmax(inference([[5.0, 3.6, 1.4, 0.2]]), 1)) # should be 1 => versicolor print sess.run(tf.argmax(inference([[5.5, 2.4, 3.8, 1.1]]), 1)) # should be 2 => virginica print sess.run(tf.argmax(inference([[6.9, 3.1, 5.1, 2.3]]), 1)) coord.request_stop() coord.join(threads) sess.closs() If you run this, it should print an accuracy value close to 1. This means our network correctly classifies the samples in almost 100% of the cases, and also we are providing the right answers for the manual samples to the model. Conclusion Our example was very simple, but TensorFlow actually allows you to do much more complicated things with similar ease, such as working with voice recognition and computer vision. It may not look much different than using any other deep learning or math packages, but the key is the ability to run the expressed model in parallel. Google is willing to create a mainstream DSL to express data algorithms focused on machine learning, and they may succeed in doing so. For instance, although Google has not yet open sourced the distributed version of the engine, a tool capable of running Tensorflow-modeled graphs directly over an Apache Spark cluster was just presented at the Spark Summit, which shows that the community is interested in expanding its usage. About the author Ariel Scarpinelli is a senior Java developer in VirtualMind and is a passionate developer with more than 15 years of professional experience. He can be found on Twitter at @ triforcexp.

Like many cloud providers, IBM is invested and interested in developing and building out their machine learning offerings. Because more and more companies are interested in trying to figure out what machine learning can currently do and what it is on the cusp of doing, there has been a plethora of new services and companies trying to see if they can either do anything new or be better than the many other competitors in the market. While much of the machine learning world is based around cloud computing and the ability to horizontally scale during training, the reality is that not every company is cloud based. While IBM has had many machine learning tools and services available on their various cloud platforms, the IBM Machine Learning system seems to be just an on-premise compromise to many of the previously cloud-based APIs that IBM offered under their machine learning and Watson brand. While I am sure many enterprise-level companies have a large amount of data that would previously not have been able to use these services, I’m unaware whether and skeptical that these services would be any better or more utilitarian than their cloud counterparts. This seems like it may be particularly useful to companies in industries with many regulations or worries about hosting data outside of their private network, although this mentality seems like it is being slowly eroded away and becoming outdated in many senses. One of the great things about the IBM Machine Learning system (and many similar companies as well) are the APIs that allow developers to pick whatever language they would like and allowing multiple frameworks because there are so many available and interesting options at the moment. This is a really important aspect for something like deep learning, where there is a constant barrage of new architectures and ideas that iterate upon prior ideas, but require new architecture and developer implementations. While I have not used IBM’s new system and will most likely not be in any particular situation where it would be available, I was lucky enough to participate in IBM’s Catalyst program and used a lot of their cloud services for a deep learning project and testing out many of their available Bluemix offerings. While I thought the system was incredibly nice and simple to use compared to many other cloud providers, I found the various machine learning APIs they offered either weren’t that useful or seemed to provide worse results than their comparable Google, Azure, and other such services. Perhaps that has changed, or their new services are much better and will be available on this new system, but it is hard to find any definitive information that this is true. One aspect of these machine learning tools that I am not excited about is the constant focus on using machine learning to create some business cost-savings models (which the IBM Machine Learning press release touts), which companies may state is passed onto the customers, but it seems like this is rarely the truth (and one of the things that was stressed on in the IBM Machine Learning launch event). The ability for machine learning methodology to solve tedious and previously complicated problems is much more important and fascinating to me than simply saving money via optimizing business expenses. While many of these problems are much harder to solve and we are far from solving them, the current applications of machine learning in business and marketing areas often provides proof for the rhetoric that machine learning is exploitive and a toxic solution. Along with this, while the IBM Machine Learning and Data Science products may seem to be aimed at someone in a data scientist role, I can never help but wonder to what extent data scientists are actually using these tools outside of pure novelty or an extremely simple prototyping step in a more in-depth analytical pipeline. I personally think the ability of these tools to create usefulness for someone who may otherwise not be interested in many aspects of traditional data science is where the tools are incredibly powerful and useful. While not all traditional developers or analysts are interested in in-depth data exploration, creating pipelines, and many other traditional data science skills, the ability to do so at ease and without having to learn skills and tooling can be seen as a huge burden to those not particularly interested. While true machine learning and data science skills are unlikely to be completely replaceable, many of the skills that a traditional data scientist has will be less important as more people are capable of doing what previously may have been a quite technical or complicated process. These sorts of products and services that are a part of the IBM Machine Learning catalog reinforce that idea and are an important step in allowing services to be used regardless of data location or analytical background, and are an important step forward for machine learning and data science in general. About the author  Graham Annett is an NLP engineer at Kip (Kipthis.com). He has been interested in deep learning and has worked with and contributed to Keras. He can be found on GitHub or on his website. 

Artificial Intelligence is just around the corner. Of course, it's been just around the corner for decades, but in part that's our own tendency to move the goalposts about what 'intelligence' is. Once, playing chess was one of the smartest things you could do. Now that a computer can easily beat a Grand Master, we've reclassified it as just standard computation, not requiring proper thinking skills. With the rise of deep learning and the proliferation of machine learning analytics, we edge ever closer to the moment where a computer system will be able to accomplish anything and everything better than a human can. So should we start worrying about SkyNet? Yes and no. Rule of the Human Overlords Early use of artificial intelligence will probably look a lot like how we used machine learning today. We'll see 'AI empowered humans' being the Human Overlords to their robot servants. These AI are smart enough to come up with the 'best options' to address human problems, but haven't been given the capability to execute them. Think about Google Maps - there, an extremely 'intelligent' artificial program comes up with the quickest route for you to take to get from point A to point B. But it doesn't force you to take it - you get to decide from the options offered which one will best suit your needs. This is likely what working alongside the first AI will look like. Rise of the Driverless Car The problem is that we are almost certainly going to see the power of AI increase exponentially - and any human greenlighting will become an increasingly inefficient part of the system. In much the same way that we'll let the Google Maps AI start to make decisions for us when we let it drive our driverless cars, we'll likely start turning more and more of our decisions over for AI to take responsibility for. Super smart AI will also likely be able to comprehend things that humans just can't understand. The mass of data that it's analysed will be beyond any one human to be able to judge effectively. Even today, financial algorithms are making instantaneous choices about the stock market - with humans just clicking 'yes' because the computer knows best. We've already seen electronic trading glitches leading to economic crises - six years ago! Just how much responsibility might we start turning over to smart machines? The Need to Solve Ethics If we've given power to an AI to make decisions for us, we'll want to ensure it has our best interests at heart, right? It's vital to program some sort of ethical system into our AI - the problem is, humans aren't very good at deciding what is and isn't ethical! Think about a simple and seemingly universal rule like 'Don't kill people'. Now think about all the ways we disagree about when it's okay to break that rule - in self-defence, in executing dangerous criminals, to end suffering, in combat. Imagine trying to code all of that into an AI, for every different moral variation. Arguably, it might be beyond human capacity. And as for right and wrong, well, we've had thousands of years of debate about that and we still can't agree exactly what is and isn't ethical. So how can we hope to program a morality system we'd be happy to give to an increasingly powerful AI? Avoiding SkyNet It may seem a little ridiculous to start worrying about the existential threat of AI when your machine learning algorithms keep bugging out on your constantly. And certainly, the possibilities offered by AI are amazing - more intelligence means faster, cheaper, and more effective solutions to humanity's problems. So despite the risk of us being outpaced by alien machine minds that have no concept of our human value system, we must always balance that risk against the amazing potential rewards. Perhaps what's most important is just not to be blase about what super-intelligent means for AI. And frankly, I can't remember how I lived before Google Maps.

The world of data has really boomed in the last few years. When I first joined Packt Hadoop was The Next Big Thing on the horizon and what people are now doing with all the data we have available to us was unthinkable. Even in the first few weeks of 2016 we’re already seeing machine learning being used in ways we probably wouldn’t have thought about even a few years ago – we’re using machine learning for everything from discovering a supernova that was 570 billion times brighter than the sun to attempting to predict this year’s Super Bowl winners based on past results, but So what else can we expect in the next year for machine learning and how will it affect us? Based on what we’ve seen over the last three years here are a few predictions about what we can expect to happen in 2016 (With maybe a little wishful thinking mixed in too!) Machine Learning becomes the new Cloud Not too long ago every business started noticing the cloud, and with it came a shift in how companies were structured. Infrastructure was radically adapted to take full advantage that the benefits that the cloud offers and it doesn’t look to be slowing down with Microsoft recently promising to spend over $1 billion in providing free cloud resources for non-profits. Starting this year it’s plausible that we’ll see a new drive to also bake machine learning into the infrastructure. Why? Because every company will want to jump on that machine learning bandwagon! The benefits and boons to every company are pretty enticing – ML offers everything from grandiose artificial intelligence to much more mundane such as improvements to recommendation engines and targeted ads; so don’t be surprised if this year everyone attempts to work out what ML can do for them and starts investing in it. The growth of MLaaS Last year we saw Machine Learning as a Service appear on the market in bigger numbers. Amazon, Google, IBM, and Microsoft all have their own algorithms available to customers. It’s a pretty logical move and why that’s not all surprising. Why? Well, for one thing, data scientists are still as rare as unicorns. Sure, universities are creating new course and training has become more common, but the fact remains we won’t be seeing the benefits of these initiatives for a few years. Second, setting up everything for your own business is going to be expensive. Lots of smaller companies simply don’t have the money to invest in their own personal machine learning systems right now, or have the time needed to fine tune it. This is where sellers are going to be putting their investments this year – the smaller companies who can’t afford a full ML experience without outside help. Smarter Security with better protection The next logical step in security is tech that can sense when there are holes in its own defenses and adapt to them before trouble strikes. ML has been used in one form or another for several years in fraud prevention, but in the IT sector we’ve been relying on static rules to detect attack patterns. Imagine if systems could detect irregular behavior accurately or set up risk scores dynamically in order to ensure users had the best protection they could at any time? We’re a long way from this being fool-proof unfortunately, but as the year progresses we can expect to see the foundations of this start being seen. After all, we’re already starting to talk about it. Machine Learning and Internet of Things combine We’re already nearly there, but with the rise in interest in the IoT we can expect that these two powerhouses to finally combine. The perfect dream for IoT hobbyists has always been like something out of the Jetsons or Wallace and Gromit –when you pass that sensor by the frame of your door in the morning your kettle suddenly springs to life so you’re able to have that morning coffee without waiting like the rest of us primals; but in truth the Internet of Things has the potential to be so much more than just making the lives of hobbyists much easier. By 2020 it is expected that over 25 billion ‘Things’ will be connected to the internet, and each one will be collating reams and reams of data. For a business with the capacity to process this data they can collect the insight they could collect is a huge boon for everything from new products to marketing strategy. For IoT to really live up to the dreams we have for it we need a system that can recognize and collate relevant data, which is where a ML system is sure to take center stage. Big things are happening in the world of machine learning, and I wouldn’t be surprised if something incredibly left field happens in the data world that takes us all by surprise, but what do you think is next for ML? If you’re looking to either start getting into the art of machine learning or boosting your skills to the next level then be sure to give our Machine Learning tech page a look; it’s filled our latest and greatest ML books and videos out right now along with the titles we’re realizing soon, available to preorder in your format of choice.

It's been nearly four months since TensorFlow, Google's computation graph machine learning library, was open sourced, and the momentum from its launch is still going strong. Over the time, both Microsoft and Baidu have released their own deep-learning libraries (CNTK and warp-ctc, respectively), and the machine learning arms race has escalated even further with Yahoo open sourcing CaffeOnSpark. Google hasn't been idle, however, and with the recent releases of TensorFlow Serving and the long awaited distributed runtime, now is the time for businesses and individual data scientists to ask: is it time to commit to TensorFlow? TensorFlow's most appealing features There are a lot of machine learning libraries available today—what makes TensorFlow stand out in this crowded space? 1. Flexibility without headaches TensorFlow heavily borrows concepts from the more tenured machine learning library Theano. Many models written for research papers were built in Theano, and its composable, node-by-node writing style translates well when implementing a model whose graph was drawn by hand first. TensorFlow's API is extremely similar. Both Theano and TensorFlow feature a Python API for defining the computation graph, which then hooks into high performance C/C++ implementations of mathematical operations. Both are able to automatically differentiate their graphs with respect to their inputs, which facilitates learning on complicated neural network structures and both integrate tightly with Numpy for defining tensors (n-dimensional arrays). However, one of the biggest advantages TensorFlow currently has over Theano (at least when comparing features both Theano and TensorFlow have) is its compile time. As of the time of writing this, Theano's compile times can be quite lengthy and although there are options to speed up compilation for experimentation, they come at the cost of a slower output model. TensorFlow's compilation is much faster, which leads to less headaches when trying out slightly different versions of models. 2. It's backed by Google (and the OSS community) At first, it may sound more like brand recognition than a tangible advantage, but when I say it's 'backed' by Google, what I mean is that Google is seriously pouring tons of resources into making TensorFlow an awesome tool. There is an entire team at Google dedicated on maintaining and improving the software steadily and visibly, while simultaneously running a clinic on how to properly interact with and engage the open source community. Google proved itself willing to adopt quality submissions from the community as well as flexible enough to adapt to public demands (such as moving the master contribution repository from Google's self-hosted Gerrit server to GitHub). These actions combined with genuinely constructive feedback from Google's team on pull-requests and issues helped make the community feel like this was a project worth supporting. The result? A continuous stream of little improvements and ideas from the community while the core Google team works on releasing larger features. Not only does TensorFlow recieve the benefits of a larger contributor base because of this, it also is more likely to withstand user decay as more people have invested time in making TensorFlow their own. 3. Easy visualizations and debugging with TensorBoard TensorBoard was the shiny toy that shipped on release with the first open source version of TensorFlow, but it's much more than eye candy. Not only can you use it as a guide to ensure what you've coded matches your reference model, but you can also keep track of data flowing through your model. This is especially useful when debugging subsections of your graph, as you can go in and see where any hiccups may have occurred. 4. TensorFlow Serving cuts the development-deployment cycle by nearly half The typical life cycle of machine learning models in the business world is generally as follows: Research and develop a model that is more accurate/faster/more descriptive than the previous model Write down the exact specifications of the finalized model Recreate the model in C++/C/Java/some other fast, compiled language Push the new model into deployment, replacing the old model Repeat On release, TensorFlow promised to "connect research and production." However, the community had to wait until just recently for that promise to come to fruition with TensorFlow Serving. This software allows you to run it as a server that can natively run models built in TensorFlow, which makes the new life cycle look like this: Research and develop a new model Hook the new model into TensorFlow Serving Repeat While there is overhead in learning how to use TensorFlow Serving, the process of hooking up new models stays the same, whereas rewriting new models in a different language is time consuming and difficult. 5. Distributed learning out of the box The distributed runtime is one of the newest features to be pushed to the TensorFlow repository, but it has been, by far, the most eagerly anticipated aspect of TensorFlow. Without having to incorporate any other libraries or software packages, TensorFlow is able to run distributed learning tasks on heterogenous hardware with various CPUs and GPUs. This feature is absolutely brand new (it came out in the middle of writing this post!), so do your research on how to use it and how well it runs. Areas to look for improvement TensorFlow can't claim to be the best at everything, and there are several sticking points that should be addressed sooner rather than later. Luckily, Google has been making steady improvements to TensorFlow since it was released, and I would be surprised if most of these were not remedied within the next few months. Runtime speed Although the TensorFlow team promises deployment worthy models from compiled TensorFlow code, at this time, its single machine training speed lags behind most other options. The team has made improvements in speed since its release, but there is still more work to be done. In-place operations, a more efficient node placement algorithm, and better compression techniques could help here. Distributed benchmarks are not available at this time—expect to see them after the next official TensorFlow release. Pre-trained models Libraries such as Caffe, Torch, and Theano have a good selection of pre-trained, state-of-the-art models that are implemented in their library. While Google did release a version of its Inception-v3 model in TensorFlow, it needs more options to provide a starting place for more types of problems. Expanded distributed support Yes, TensorFlow did push code for it's distributed runtime, but it still needs better documentation as well as more examples. I'm incredibly excited that it's available to try out right now, but it's going to take some time for most people to put it into production. Interested in getting up and running with TensorFlow? You'll need a primer on Python. Luckily, our Python Fundamentals course in Mapt gives you an accessible yet comprehensive journey through Python - and this week it's completely free. Click here, login, then get stuck in... The future Most people want to use software that is going to last for more than a few months—what does the future look like for TensorFlow? Here are my predictions about the medium-term future of the library. Enterprise-level distributions Just as Hadoop has commercial distributions of its software, I expect to see more and more companies offering supported suites that tie into TensorFlow. Whether they have more pre-trained models built on top of Keras (which already supports a TensorFlow backend), or make TensorFlow work seamlessly with a distributed file system like Hadoop, I forsee a lot of demand for enterprise features and support with TensorFlow. TensorFlow's speed will catch up (and most users won't need it) As mentioned earlier, TensorFlow still lags behind many other libraries out there. However, with the improvements already made; it's clear that Google is determined to make TensorFlow as efficient as possible. That said, I believe most applications of TensorFlow won't desperately need the speed increase. Of course, it's nice to have your models run faster, but most businesses out there don't have petabytes of useful data to work with, which means that model training usually doesn't take the "weeks" that we often see claimed as training time. TensorFlow is going to get easier, not more difficult, over time While there are definitely going to be many new features in upcoming releases of TensorFlow, I expect to see the learning curve of the software go down as more resources, such as tutorials, examples, and books are made available. The documentation's terminology has already changed in places to be more understandable; navigation within the documentation should improve over time. Finally, while most of the latest features in TensorFlow don't have the friendliest APIs right now, I'd be shocked if more user-friendly versions of TensorFlow Serving and the distributed runtime weren't in the works right now. Should I use TensorFlow? TensorFlow appears primed to fulfil the promise that was made back in November: a distributed, flexible data flow graph library that excels at neural network composition. I leave it to you decision makers to figure out whether TensorFlow is the right move for your own machine learning tasks, but here is my overall impression of TensorFlow: no other machine learning framework targeted at production-level tasks is as flexible, powerful, or improving as rapidly as TensorFlow. While other frameworks may carry advantages over TensorFlow now, Google is putting the effort into making consistent improvements, which bodes well for a community that is still in its infancy. About the author Sam Abrahams is a freelance data engineer and animator in Los Angeles, CA. He specializes in real-world applications of machine learning and is a contributor to TensorFlow. Sam runs a small tech blog, Memdump, and is an active member of the local hacker scene in West LA.

The FAT* 2018 Conference on Fairness, Accountability, and Transparency is a first-of-its-kind international and interdisciplinary peer-reviewed conference that seeks to publish and present work examining the fairness, accountability, and transparency of algorithmic systems. This article covers research papers dedicated to 1st Session on Online discrimination and Privacy. FAT*  hosted the presentation of research work from a wide variety of disciplines, including computer science, statistics, the social sciences, and law. It took place on February 23 and 24, 2018, at the New York University Law School, in cooperation with its Technology Law and Policy Clinic. The conference brought together over 450 attendees from academic researchers, policymakers, and Machine learning practitioners. It witnessed 17 research papers, 6 tutorials, and 2 keynote presentations from leading experts in the field.  Session 1 explored ways in which online discrimination can happen and privacy could be compromised. The papers presented look for novel and practical solutions to some of the problems identified. We attempt to introduce our readers to the papers that will be presented at FAT* 2018 in this area thereby summarising the key challenges and questions explored by leading minds on the topic and their proposed potential answers to those issues. Session Chair: Joshua Kroll (University of California, Berkeley) Paper 1: Potential for Discrimination in Online Targeted Advertising Problems identified in the paper: Much recent work has focused on detecting instances of discrimination in online services ranging from discriminatory pricing on e-commerce and travel sites like Staples (Mikians et al., 2012) and Hotels.com (Hannák et al., 2014) to discriminatory prioritization of service requests and offerings from certain users over others in crowdsourcing and social networking sites like TaskRabbit (Hannák et al., 2017). In this paper, we focus on the potential for discrimination in online advertising, which underpins much of the Internet’s economy. Specifically, we focus on targeted advertising, where ads are shown only to a subset of users that have attributes (features) selected by the advertiser. Key Takeaways: A malicious advertiser can create highly discriminatory ads without using sensitive attributes such as gender or race. The current methods used to counter the problem are insufficient. The potential for discrimination in targeted advertising arises from the ability of an advertiser to use the extensive personal (demographic, behavioral, and interests) data that ad platforms gather about their users to target their ads. Different targeting methods offered by Facebook: attribute-based targeting, PII-based (custom audience) targeting and Look-alike audience targeting Three basic approaches to quantifying discrimination and their tradeoffs: Based on advertiser’s intent Based on ad targeting process Based on the targeted audience (outcomes) Paper 2: Discrimination in Online Personalization: A Multidisciplinary Inquiry The authors explore ways in which discrimination may arise in the targeting of job-related advertising, noting the potential for multiple parties to contribute to its occurrence. They then examine the statutes and case law interpreting the prohibition on advertisements that indicate a preference based on protected class and consider its application to online advertising. This paper provides a legal analysis of a real case, which found that simulated users selecting a gender in Google’s Ad Settings produces employment-related advertisements differing rates along gender lines despite identical web browsing patterns. Key Takeaways: The authors’ analysis of existing case law concludes that Section 230 may not immunize advertising platforms from liability under the FHA for algorithmic targeting of advertisements that indicate a preference for or against a protected class. Possible causes of ad targeting: Targeting was fully a product of the advertiser selecting gender segmentation. Targeting was fully a product of machine learning—Google alone selects gender. Targeting was fully a product of the advertiser selecting keywords. Targeting was fully the product of the advertiser being outbid for women. Given the limited scope of Title VII the authors conclude that Google is unlikely to face liability on the facts presented by Datta et al. Thus, the advertising prohibition of Title VII, like the prohibitions on discriminatory employment practices, is ill-equipped to advance the aims of equal treatment in a world where algorithms play an increasing role in decision making. Paper 3: Privacy for All: Ensuring Fair and Equitable Privacy Protections In this position paper, the authors argue for applying recent research on ensuring sociotechnical systems are fair and non-discriminatory to the privacy protections those systems may provide. Just as algorithmic decision-making systems may have discriminatory outcomes even without explicit or deliberate discrimination, so also privacy regimes may disproportionately fail to protect vulnerable members of their target population, resulting in disparate impact with respect to the effectiveness of privacy protections. Key Takeaways: Research questions posed: Are technical or non-technical privacy protection schemes fair? When and how do privacy protection technologies or policies improve or impede the fairness of systems they affect? When and how do fairness-enhancing technologies or policies enhance or reduce the privacy protections of the people involved? Data linking can lead to deanonymization; live recommenders can also be attacked to leak information The authors propose a new definition for a fair privacy scheme: a privacy scheme is (group-)fair if the probability of failure and expected risk are statistically independent of the subject’s membership in a protected class.   If you have missed Session 2, Session 3, Session 4 and Session 5 of the FAT* 2018 Conference, we have got you covered.

If you've been looking into the world of Machine Learning lately you might have heard about a mysterious thing called “Deep Learning”. But just what is Deep Learning, and what does it mean for the world of Machine Learning as a whole? Take less than two minutes out of your day to find out and fully realize the awesome potential Deep Learning has with this video today. Plus, if you’re already in love with Deep Learning, or want to finally start your Deep Learning journey then be sure to pick up one of our recommendations below and get started right now.