How-To Tutorials

In this article by Artemij Fedosejev, the author of React.js Essentials, we will take a look at test suites, specs, and expectations. To write a test for JavaScript functions, you need a testing framework. Fortunately, Facebook built their own unit test framework for JavaScript called Jest. It is built on top of Jasmine - another well-known JavaScript test framework. If you’re familiar with Jasmine you’ll find Jest's approach to testing very similar. However I'll make no assumptions about your prior experience with testing frameworks and discuss the basics first. The fundamental idea of unit testing is that you test only one piece of functionality in your application that usually is implemented by one function. And you test it in isolation - meaning that all other parts of your application which that function depends on are not used by your tests. Instead, they are imitated by your tests. To imitate a JavaScript object is to create a fake one that simulates the behavior of the real object. In unit testing the fake object is called mock and the process of creating it is called mocking. Jest automatically mocks dependencies when you're running your tests. Better yet, it automatically finds tests to execute in your repository. Let's take a look at the example. Create a directory called ./snapterest/source/js/utils/ and create a new file called TweetUtils.js within it, with the following contents: function getListOfTweetIds(tweets) {  return Object.keys(tweets);}module.exports.getListOfTweetIds = getListOfTweetIds; TweetUtils.js file is a module with the getListOfTweetIds() utility function for our application to use. Given an object with tweets, getListOfTweetIds() returns an array of tweet IDs. Using the CommonJS module pattern we export this function: module.exports.getListOfTweetIds = getListOfTweetIds; Jest Unit Testing Now let's write our first unit test with Jest. We'll be testing our getListOfTweetIds() function. Create a new directory: ./snapterest/source/js/utils/__tests__/. Jest will run any tests in any __tests__ directories that it finds within your project structure. So it's important to name your directories with tests: __tests__. Create a TweetUtils-test.js file inside of __tests__:jest.dontMock('../TweetUtils');describe('Tweet utilities module', function () {  it('returns an array of tweet ids', function () {    var TweetUtils = require('../TweetUtils');    var tweetsMock = {      tweet1: {},      tweet2: {},      tweet3: {}    };    var expectedListOfTweetIds = ['tweet1', 'tweet2', 'tweet3'];    var actualListOfTweetIds = TweetUtils.getListOfTweetIds(tweetsMock);    expect(actualListOfTweetIds).toBe(expectedListOfTweetIds);  });}); First we tell Jest not to mock our TweetUtils module: jest.dontMock('../TweetUtils'); We do this because Jest will automatically mock modules returned by the require() function. In our test we're requiring the TweetUtils module: var TweetUtils = require('../TweetUtils'); Without the jest.dontMock('../TweetUtils') call, Jest would return an imitation of our TweetUtils module, instead of the real one. But in this case we actually need the real TweetUtils module, because that's what we're testing. Creating test suites Next we call a global Jest function describe(). In our TweetUtils-test.js file we're not just creating a single test, instead we're creating a suite of tests. A suite is a collection of tests that collectively test a bigger unit of functionality. For example a suite can have multiple tests which tests all individual parts of a larger module. In our example, we have a TweetUtils module with a number of utility functions. In that situation we would create a suite for the TweetUtils module and then create tests for each individual utility function, like getListOfTweetIds(). describe defines a suite and takes two parameters: Suite name - the description of what is being tested: 'Tweet utilities module'. Suit implementation: the function that implements this suite. In our example, the suite is: describe('Tweet utilities module', function () {  // Suite implementation goes here...}); Defining specs How do you create an individual test? In Jest, individual tests are called specs. They are defined by calling another global Jest function it(). Just like describe(), it() takes two parameters: Spec name: the title that describes what is being tested by this spec: 'returns an array of tweet ids'. Spec implementation: the function that implements this spec. In our example, the spec is: it('returns an array of tweet ids', function () {  // Spec implementation goes here...}); Let's take a closer look at the implementation of our spec: var TweetUtils = require('../TweetUtils');var tweetsMock = {  tweet1: {},  tweet2: {},  tweet3: {}};var expectedListOfTweetIds = ['tweet1', 'tweet2', 'tweet3'];var actualListOfTweetIds = TweetUtils.getListOfTweetIds(tweetsMock);expect(actualListOfTweetIds).toEqual(expectedListOfTweetIds); This spec tests whether getListOfTweetIds() method of our TweetUtils module returns an array of tweet IDs when given an object with tweets. First we import the TweetUtils module: var TweetUtils = require('../TweetUtils'); Then we create a mock object that simulates the real tweets object: var tweetsMock = {  tweet1: {},  tweet2: {},  tweet3: {}}; The only requirement for this mock object is to have tweet IDs as object keys. The values are not important hence we choose empty objects. Key names are not important either, so we can name them tweet1, tweet2 and tweet3. This mock object doesn't fully simulate the real tweet object. Its sole purpose is to simulate the fact that its keys are tweet IDs. The next step is to create an expected list of tweet IDs: var expectedListOfTweetIds = ['tweet1', 'tweet2', 'tweet3']; We know what tweet IDs to expect because we've mocked a tweets object with the same IDs. The next step is to extract the actual tweet IDs from our mocked tweets object. For that we use getListOfTweetIds()that takes the tweets object and returns an array of tweet IDs: var actualListOfTweetIds = TweetUtils.getListOfTweetIds(tweetsMock); We pass tweetsMock to that method and store the results in actualListOfTweetIds. The reason this variable is named actualListOfTweetIds is because this list of tweet IDs is produced by the actual getListOfTweetIds() function that we're testing. Setting Expectations The final step will introduce us to a new important concept: expect(actualListOfTweetIds).toEqual(expectedListOfTweetIds); Let's think about the process of testing. We need to take an actual value produced by the method that we're testing - getListOfTweetIds(), and match it to the expected value that we know in advance. The result of that match will determine if our test has passed or failed. The reason why we can guess what getListOfTweetIds() will return in advance is because we've prepared the input for it - that's our mock object: var tweetsMock = {  tweet1: {},  tweet2: {},  tweet3: {}}; So we can expect the following output from calling TweetUtils.getListOfTweetIds(tweetsMock): ['tweet1', 'tweet2', 'tweet3'] But because something can go wrong inside of getListOfTweetIds() we cannot guarantee this result - we can only expect it. That's why we need to create an expectation. In Jest, an Expectation is built using expect()which takes an actual value, for example: actualListOfTweetIds. expect(actualListOfTweetIds) Then we chain it with a Matcher function that compares the actual value with the expected value and tells Jest whether the expectation was met. expect(actualListOfTweetIds).toEqual(expectedListOfTweetIds); In our example we use the toEqual() matcher function to compare two arrays. Click here for a list of all built-in matcher functions in Jest. And that's how you create a spec. A spec contains one or more expectations. Each expectation tests the state of your code. A spec can be either a passing spec or a failing spec. A spec is a passing spec only when all expectations are met, otherwise it's a failing spec. Well done, you've written your first testing suite with a single spec that has one expectation. Continue reading React.js Essentials to continue your journey into testing.

Since Rust 1.0 has a great macro system, it allows us to apply some code to multiple types or expressions, as they work by expanding themselves at compile time. This means that when you use a macro, you are effectively writing a lot of code before the actual compilation starts. This has two main benefits, first, the codebase can be easier to maintain by being smaller and reusing code. Second, since macros expand before starting the creation of object code, you can abstract at the syntactic level. In this article, we'll learn how to create our very own macros in Rust. This Rust tutorial is an extract from Rust High Performance, authored by Iban Eguia Moraza. For example, you can have a function like this one: fn add_one(input: u32) -> u32 { input + 1 } This function restricts the input to u32 types and the return type to u32. We could add some more accepted types by using generics, which may accept &u32 if we use the Add trait. Macros allow us to create this kind of code for any element that can be written to the left of the + sign and it will be expanded differently for each type of element, creating a different code for each case. To create a macro, you will need to use a macro built into the language, the macro_rules!{} macro. This macro receives the name of the new macro as a first parameter and a block with the macro code as a second element. The syntax can be a bit complex the first time you see it, but it can be learned quickly. Let's start with a macro that does just the same as the function we saw before: macro_rules! add_one { ($input:expr) => { $input + 1 } } You can now call that macro from your main() function by calling add_one!(integer);. Note that the macro needs to be defined before the first call, even if it's in the same file. It will work with any integer, which wasn't possible with functions. Let's analyze how the syntax works. In the block after the name of the new macro (add_one), we can see two sections. In the first part, on the left of the =>, we see $input:expr inside parentheses. Then, to the right, we see a Rust block where we do the actual addition. The left part works similarly (in some ways) to a pattern match. You can add any combination of characters and then some variables, all of them starting with a dollar sign ($) and showing the type of variable after a colon. In this case, the only variable is the $input variable and it's an expression. This means that you can insert any kind of expression there and it will be written in the code to the right, substituting the variable with the expression. Creating Macro variants As you can see, it's not as complicated as you might think. As I wrote, you can have almost any pattern to the left of the macro_rules!{} side. Not only that, you can also have multiple patterns, as if it were a match statement, so that if one of them matches, it will be the one expanded. Let's see how this works by creating a macro which, depending on how we call it, will add one or two to the given integer: macro_rules! add { {one to $input:expr} => ($input + 1); {two to $input:expr} => ($input + 2); } fn main() { println!("Add one: {}", add!(one to 25/5)); println!("Add two: {}", add!(two to 25/5)); } You can see a couple of clear changes to the macro. First, we swapped braces for parentheses and parentheses for braces in the macro. This is because in a macro, you can use interchangeable braces ({ and }), square brackets ([ and ]), and parentheses (( and )). Not only that, you can use them when calling the macro. You have probably already used the vec![] macro and the format!() macro, and we saw the lazy_static!{} macro in the last chapter. We use brackets and parentheses here just for convention, but we could call the vec!{} or the format![] macros the same way, because we can use braces, brackets, and parentheses in any macro call. The second change was to add some extra text to our left-hand side patterns. We now call our macro by writing the text one to or two to, so I also removed the one redundancy to the macro name and called it add!(). This means that we now call our macro with literal text. That is not valid Rust, but since we are using a macro, we modify the code we are writing before the compiler tries to understand actual Rust code and the generated code is valid. We could add any text that does not end the pattern (such as parentheses or braces) to the pattern. The final change was to add a second possible pattern. We can now add one or two and the only difference will be that the right side of the macro definition must now end with a trailing semicolon for each pattern (the last one is optional) to separate each of the options. A small detail that I also added in the example was when calling the macro in the main() function. As you can see, I could have added one or two to 5, but I wrote 25/5 for a reason. When compiling this code, this will be expanded to 25/5 + 1 (or 2, if you use the second variant). This will later be optimized at compile time, since it will know that 25/5 + 1 is 6, but the compiler will receive that expression, not the final result. The macro system will not calculate the result of the expression; it will simply copy in the resulting code whatever you give to it and then pass it to the next compiler phase. You should be especially careful with this when a macro you are creating calls another macro. They will get expanded recursively, one inside the other, so the compiler will receive a bunch of final Rust code that will need to be optimized. Issues related to this were found in the CLAP crate that we saw in the last chapter, since the exponential expansions were adding a lot of bloat code to their executables. Once they found out that there were too many macro expansions inside the other macros and fixed it, they reduced the size of their binary contributions by more than 50%. Macros allow for an extra layer of customization. You can repeat arguments more than once. This is common, for example, in the vec![] macro, where you create a new vector with information at compile time. You can write something like vec![3, 4, 76, 87];. How does the vec![] macro handle an unspecified number of arguments? Creating Complex macros We can specify that we want multiple expressions in the left-hand side pattern of the macro definition by adding a * for zero or more matches or a + for one or more matches. Let's see how we can do that with a simplified my_vec![] macro: macro_rules! my_vec { ($($x: expr),*) => {{ let mut vector = Vec::new(); $(vector.push($x);)* vector }} } Let's see what is happening here. First, we see that on the left side, we have two variables, denoted by the two $ signs. The first makes reference to the actual repetition. Each comma-separated expression will generate a $x variable. Then, on the right side, we use the various repetitions to push $x to the vector once for every expression we receive. There is another new thing on the right-hand side. As you can see, the macro expansion starts and ends with a double brace instead of using only one. This is because, once the macro gets expanded, it will substitute the given expression for a new expression: the one that gets generated. Since what we want is to return the vector we are creating, we need a new scope where the last sentence will be the value of the scope once it gets executed. You will be able to see it more clearly in the next code snippet. We can call this code with the main() function: fn main() { let my_vector = my_vec![4, 8, 15, 16, 23, 42]; println!("Vector test: {:?}", my_vector); } It will be expanded to this code: fn main() { let my_vector = { let mut vector = Vec::new(); vector.push(4); vector.push(8); vector.push(15); vector.push(16); vector.push(23); vector.push(42); vector }; println!("Vector test: {:?}", my_vector); } As you can see, we need those extra braces to create the scope that will return the vector so that it gets assigned to the my_vector binding. You can have multiple repetition patterns on the left expression and they will be repeated for every use, as needed on the right. macro_rules! add_to_vec { ($( $x:expr; [ $( $y:expr ),* ]);* ) => { &[ $($( $x + $y ),*),* ] } } In this example, the macro can receive one or more $x; [$y1, $y2,...] input. So, for each input, it will have one expression, then a semicolon, then a bracket with multiple sub-expressions separated by a comma, and finally, another bracket and a semicolon. But what does the macro do with this input? Let's check to the right-hand side of it. As you can see, this will create multiple repetitions. We can see that it creates a slice (&[T]) of whatever we feed to it, so all the expressions we use must be of the same type. Then, it will start iterating over all $x variables, one per input group. So if we feed it only one input, it will iterate once for the expression to the left of the semicolon. Then, it will iterate once for every $y expression associated with the $x expression, add them to the + operator, and include the result in the slice. If this was too complex to understand, let's look at an example. Let's suppose we call the macro with 65; [22, 34] as input. In this case, 65 will be $x, and 22, 24, and so on will be $y variables associated with 65. So, the result will be a slice like this: &[65+22, 65+34]. Or, if we calculate the results: &[87, 99]. If, on the other hand, we give two groups of variables by using 65; [22, 34]; 23; [56, 35] as input, in the first iteration, $x will be 65, while in the second one, it will be 23. The $y variables of 64 will be 22 and 34, as before, and the ones associated with 23 will be 56 and 35. This means that the final slice will be &[87, 99, 79, 58], where 87 and 99 work the same way as before and 79 and 58 are the extension of adding 23 to 56 and 23 to 35. This gives you much more flexibility than the functions, but remember, all this will be expanded during compile time, which can make your compilation time much slower and the final codebase larger and slower still if the macro used duplicates too much code. In any case, there is more flexibility to it yet. So far, all variables have been of the expr kind. We have used this by declaring $x:expr and $y:expr but, as you can imagine, there are other kinds of macro variables. The list follows: expr: Expressions that you can write after an = sign, such as 76+4 or if a==1 {"something"} else {"other thing"}. ident: An identifier or binding name, such as foo or bar. path: A qualified path. This will be a path that you could write in a use sentence, such as foo::bar::MyStruct or foo::bar::my_func. ty: A type, such as u64 or MyStruct. It can also be a path to the type. pat: A pattern that you can write at the left side of an = sign or in a match expression, such as Some(t) or (a, b, _). stmt: A full statement, such as a let binding like let a = 43;. block: A block element that can have multiple statements and a possible expression between braces, such as {vec.push(33); vec.len()}. item: What Rust calls items. For example, function or type declarations, complete modules, or trait definitions. meta: A meta element, which you can write inside of an attribute (#[]). For example, cfg(feature = "foo"). tt: Any token tree that will eventually get parsed by a macro pattern, which means almost anything. This is useful for creating recursive macros, for example. As you can imagine, some of these kinds of macro variables overlap and some of them are just more specific than the others. The use will be verified on the right-hand side of the macro, in the expansion, since you might try to use a statement where an expression must be used, even though you might use an identifier too, for example. There are some extra rules, too, as we can see in the Rust documentation (https://doc.rust-lang.org/book/first-edition/macros.html#syntactic-requirements). Statements and expressions can only be followed by =>, a comma, or a semicolon. Types and paths can only be followed by =>, the as or where keywords, or any commas, =, |, ;, :, >, [, or {. And finally, patterns can only be followed by =>, the if or in keywords, or any commas, =, or |. Let's put this in practice by implementing a small Mul trait for a currency type we can create. This is an adapted example of some work we did when creating the Fractal Credits digital currency. In this case, we will look to the implementation of the Amount type (https://github.com/FractalGlobal/utils-rs/blob/49955ead9eef2d9373cc9386b90ac02b4d5745b4/src/amount.rs#L99-L102), which represents a currency amount. Let's start with the basic type definition: #[derive(Copy, Clone, PartialEq, Eq, PartialOrd, Ord)] pub struct Amount { value: u64, } This amount will be divisible by up to three decimals, but it will always be an exact value. We should be able to add an Amount to the current Amount, or to subtract it. I will not explain these trivial implementations, but there is one implementation where macros can be of great help. We should be able to multiply the amount by any positive integer, so we should implement the Mul trait for u8, u16, u32, and u64 types. Not only that, we should be able to implement the Div and the Rem traits, but I will leave those out, since they are a little bit more complex. You can check them in the implementation linked earlier. The only thing the multiplication of an Amount with an integer should do is to multiply the value by the integer given. Let's see a simple implementation for u8: use std::ops::Mul; impl Mul<u8> for Amount { type Output = Self; fn mul(self, rhs: u8) -> Self::Output { Self { value: self.value * rhs as u64 } } } impl Mul<Amount> for u8 { type Output = Amount; fn mul(self, rhs: Amount) -> Self::Output { Self::Output { value: self as u64 * rhs.value } } } As you can see, I implemented it both ways so that you can put the Amount to the left and to the right of the multiplication. If we had to do this for all integers, it would be a big waste of time and code. And if we had to modify one of the implementations (especially for Rem functions), it would be troublesome to do it in multiple code points. Let's use macros to help us. We can define a macro, impl_mul_int!{}, which will receive a list of integer types and then implement the Mul trait back and forward between all of them and the Amount type. Let's see: macro_rules! impl_mul_int { ($($t:ty)*) => ($( impl Mul<$t> for Amount { type Output = Self; fn mul(self, rhs: $t) -> Self::Output { Self { value: self.value * rhs as u64 } } } impl Mul<Amount> for $t { type Output = Amount; fn mul(self, rhs: Amount) -> Self::Output { Self::Output { value: self as u64 * rhs.value } } } )*) } impl_mul_int! { u8 u16 u32 u64 usize } As you can see, we specifically ask for the given elements to be types and then we implement the trait for all of them. So, for any code that you want to implement for multiple types, you might as well try this approach, since it will save you from writing a lot of code and it will make it more maintainable. If you found this article useful and would like to learn more such tips, head on over to pick up the book, Rust High Performance, authored by Iban Eguia Moraza. Perform Advanced Programming with Rust Rust 1.28 is here with global allocators, nonZero types and more Eclipse IDE's Photon release will support Rust

DeepMind, a London based artificial intelligence (AI) company currently owned by Alphabet, recently made great strides in AI with its AlphaGo program. It all began in October 2015 when the program beat the European Go champion Fan Hui 5-0, in a game of Go. This was the very first time an AI defeated a professional Go player. Earlier, computers were only known to have played Go at the "amateur" level. Then, the company made headlines again in 2016 after its AlphaGo program beat Lee Sedol, a professional Go player (a world champion) with a score of 4-1 in a five-game match. Furthermore, in late 2017, an improved version of the program called AlphaGo Zero defeated AlphaGo 100 games to 0. The best part? AlphaGo Zero's strategies were self-taught i.e it was trained without any data from human games. AlphaGo Zero was able to defeat its predecessor in only three days time with lesser processing power than AlphaGo. However, the original AlphaGo, on the other hand required months to learn how to play. All these facts beg the questions: what makes AlphaGo Zero so exceptional? Why is it such a big deal? How does it even work? So, without further ado, let’s dive into the what, why, and how of DeepMind’s AlphaGo Zero. What is DeepMind AlphaGo Zero? Simply put, AlphaGo Zero is the strongest Go program in the world (with the exception of AlphaZero). As mentioned before, it monumentally outperforms all previous versions of AlphaGo. Just check out the graph below which compares the Elo rating of the different versions of AlphaGo. Source: DeepMind The Elo rating system is a method for calculating the relative skill levels of players in zero-sum games such as chess and Go. It is named after its creator Arpad Elo, a Hungarian-American physics professor. Now, all previous versions of AlphaGo were trained using human data. The previous versions learned and improved upon the moves played by human experts/professional Go players. But AlphaGo Zero didn’t use any human data whatsoever. Instead, it had to learn completely from playing against itself. According to DeepMind's Professor David Silver, the reason that playing against itself enables it to do so much better than using strong human data is that AlphaGo always has an opponent of just the right level. So it starts off extremely naive, with perfectly random play. And yet at every step of the learning process, it has an opponent (a “sparring partner”) that’s exactly calibrated to its current level of performance. That is, to begin with, these players are terribly weak but over time they become progressively stronger and stronger. Why is reinforcement learning such a big deal? People tend to assume that machine learning is all about big data and massive amounts of computation. But actually, with AlphaGo Zero, AI scientists at DeepMind realized that algorithms matter much more than the computing processing power or data availability. AlphaGo Zero required less computation than previous versions and yet it was able to perform at a much higher level due to using much more principled algorithms than before. It is a system which is trained completely from scratch, starting from random behavior, and progressing from first principles to really discover tabula rasa, in playing the game of Go. It is, therefore, no longer constrained by the limits of human knowledge. Note that AlphaGo Zero did not use zero-shot learning which essentially is the ability of the machine to solve a task despite not having received any training for that task. How does it work? AlphaGo Zero is able to achieve all this by employing a novel form of reinforcement learning, in which AlphaGo Zero becomes its own teacher. As explained previously, the system starts off with a single neural network that knows absolutely nothing about the game of Go. By combining this neural network with a powerful search algorithm, it then plays games against itself. As it plays more and more games, the neural network is updated and tuned to predict moves, and even the eventual winner of the games. This revised neural network is then recombined with the search algorithm to generate a new, stronger version of AlphaGo Zero, and the process repeats. With each iteration, the performance of the system enhances with each iteration, and the quality of the self-play games’ advances, leading to increasingly accurate neural networks and ever-more powerful versions of AlphaGo Zero. Now, let’s dive into some of the technical details that make this version of AlphaGo so much better than all its forerunners. AlphaGo Zero's neural network was trained using TensorFlow, with 64 GPU workers and 19 CPU parameter servers. Only four Tensor Processing Units (TPUs) were used for inference. And of course, the neural network initially knew nothing about Go beyond the rules. Both AlphaGo and AlphaGo Zero took a general approach to play Go. Both evaluated the Go board and chose moves using a combination of two methods: Conducting a “lookahead” search: This means looking ahead several moves by simulating games, and hence seeing which current move is most likely to lead to a “good” position in the future. Assessing positions based on an “intuition” of whether a position is “good” or “bad”  and is likely to result in a win or a loss. Go is a truly intricate game which means computers can’t merely search all possible moves using a brute force approach to discover the best one. Method 1: Lookahead Before AlphaGo, all the finest Go programs tackled this issue by using “Monte Carlo Tree Search” or MCTS. This process involves initially exploring numerous possible moves on the board and then focusing this search over time as certain moves are found to be more likely to result in wins than others. Source: LOC Both AlphaGo and AlphaGo Zero apply a fairly elementary version of MCTS for their “lookahead” to correctly maintain the tradeoff between exploring new sequences of moves or more deeply explore already-explored sequences. Although MCTS has been at the heart of all effective Go programs preceding AlphaGo, it was DeepMind’s smart coalescence of this method with a neural network-based “intuition” that enabled it to attain superhuman performance. Method 2: Intuition DeepMind’s pivotal innovation with AlphaGo was to utilize deep neural networks to identify the state of the game and then use this knowledge to effectively guide the search of the MCTS. In particular, they trained networks that could record: The current board position Which player was playing The sequence of recent moves (in order to rule out certain moves as “illegal”) With this data, the neural networks could propose: Which move should be played If the current player is likely to win or not So how did DeepMind train neural networks to do this? Well, AlphaGo and AlphaGo Zero used rather different approaches in this case. AlphaGo had two separately trained neural networks: Policy Network and Value Network. Source: AlphaGo’s Nature Paper DeepMind then fused these two neural networks with MCTS  —  that is, the program’s “intuition” with its brute force “lookahead” search — in an ingenious way. It used the network that had been trained to predict: Moves to guide which branches of the game tree to search Whether a position was “winning” to assess the positions it encountered during its search This let AlphaGo to intelligently search imminent moves and eventually beat the world champion Lee Sedol. AlphaGo Zero, however, took this principle to the next level. Its neural network’s “intuition” was trained entirely differently from that of AlphaGo. More specifically: The neural network was trained to play moves that exhibited the improved evaluations from performing the “lookahead” search The neural network was tweaked so that it was more likely to play moves like those that led to wins and less likely to play moves similar to those that led to losses during the self-play games Much was made of the fact that no games between humans were used to train AlphaGo Zero. Thus, for a given state of a Go agent, it can constantly be made smarter by performing MCTS-based lookahead and using the results of that lookahead to upgrade the agent. This is how AlphaGo Zero was able to perpetually improve, from when it was an “amateur” all the way up to when it better than the best human players. Moreover, AlphaGo Zero’s neural network architecture can be referred to as a “two-headed” architecture. Source: Hacker Noon Its first 20 layers were “blocks” of a typically seen in modern neural net architectures. These layers were followed by two “heads”: One head that took the output of the first 20 layers and presented probabilities of the Go agent making certain moves Another head that took the output of the first 20 layers and generated a probability of the current player winning. What’s more, AlphaGo Zero used a more “state of the art” neural network architecture as opposed to AlphaGo. Particularly, it used a “residual” neural network architecture rather than a plainly “convolutional” architecture. Deep residual learning was pioneered by Microsoft Research in late 2015, right around the time work on the first version of AlphaGo would have been concluded. So, it is quite reasonable that DeepMind did not use them in the initial AlphaGo program. Notably, each of these two neural network-related acts —  switching from separate-convolutional to the more advanced dual-residual architecture and using the “two-headed” neural network architecture instead of separate neural networks  —  would have resulted in nearly half of the increase in playing strength as was realized when both were coupled. Source: AlphaGo’s Nature Paper Wrapping it up According to DeepMind: “After just three days of self-play training, AlphaGo Zero emphatically defeated the previously published version of AlphaGo - which had itself defeated 18-time world champion Lee Sedol - by 100 games to 0. After 40 days of self-training, AlphaGo Zero became even stronger, outperforming the version of AlphaGo known as “Master”, which has defeated the world's best players and world number one Ke Jie. Over the course of millions of AlphaGo vs AlphaGo games, the system progressively learned the game of Go from scratch, accumulating thousands of years of human knowledge during a period of just a few days. AlphaGo Zero also discovered new knowledge, developing unconventional strategies and creative new moves that echoed and surpassed the novel techniques it played in the games against Lee Sedol and Ke Jie.” Further, the founder and CEO of DeepMind, Dr. Demis Hassabis believes AlphaGo's algorithms are likely to most benefit to areas that need an intelligent search through an immense space of possibilities. Author Bio Gaurav is a Senior SEO and Content Marketing Analyst at The 20 Media, a Content Marketing agency that specializes in data-driven SEO. He has more than seven years of experience in Digital Marketing and along with that loves to read and write about AI, Machine Learning, Data Science and much more about the emerging technologies. In his spare time, he enjoys watching movies and listening to music. Connect with him on Twitter and LinkedIn. DeepMind researchers provide theoretical analysis on recommender system, ‘echo chamber’ and ‘filter bubble effect’ What if AIs could collaborate using human-like values? DeepMind researchers propose a Hanabi platform. Google DeepMind’s AI AlphaStar beats StarCraft II pros TLO and MaNa; wins 10-1 against the gamers  

[box type="note" align="" class="" width=""]The following excerpt is taken from the book MySQL 8 Administrator’s Guide, co-authored by Chintan Mehta, Ankit Bhavsar, Hetal Oza and Subhash Shah. This book presents an in-depth view of the newly released features of MySQL 8 and how you can leverage them to administer a high-performance MySQL solution.[/box] Following the best practices for the configuration of MySQL helps us design and manage efficient database, and are quite a cherry on top - without which, it might seem a bit incomplete. In addition to configuration, benchmarking helps us validate and find bottlenecks in the database system and address them. In this article, we look at specific areas that will help us understand the best practices for configuration and performance benchmarking. 1. Resource utilization IO activity, CPU, and memory usage is something that you should not miss out. These metrics help us know how the system is performing while doing benchmarking and at the time of scaling. It also helps us derive impacts per transaction. 2. Stretching your benchmarking timelines We may often like to have a quick glance at performance metrics; however, ensuring that MySQL behaves in the same way for a longer duration of testing is also a key element. There is some basic stuff that might impact on performance when you stretch your benchmark timelines, such as memory fragmentation, degradation of IO, impact after data accumulation, cache management, and so on. We don't want our database to get restarted just to clean up junk items, correct? Therefore, it is suggested to run benchmarking for a long duration for stability and performance Validation. 3. Replicating production settings Let's benchmark in a production-replicated environment. Wait! Let's disable database replication in a replica environment until we are done with benchmarking. Gotcha! We have got some good numbers! It often happens that we don't simulate everything completely that we are going to configure in the production environment. It could prove to be costly, as we might unintentionally be benchmarking something in an environment that might have an adverse impact when it's in production. Replicate production settings, data, workload, and so on in your replicated environment while you do benchmarking. 4. Consistency of throughput and latency Throughput and latency go hand in hand. It is important to keep your eyes primarily focused on throughput; however, latency over time might be something to look out for. Performance dips, slowness, or stalls were noticed in InnoDB in its earlier days. It has improved a lot since then, but as there might be other cases depending on your workload, it is always good to keep an eye on throughput along with latency. 5. Sysbench can do more Sysbench is a wonderful tool to simulate your workloads, whether it be thousands of tables, transaction intensive, data in-memory, and so on. It is a splendid tool to simulate and gives you nice representation. 6. Virtualization world I would like to keep this simple; bare metal as compared to virtualization isn't the same. Hence, while doing benchmarking, measure your resources according to your environment. You might be surprised to see the difference in results if you compare both. 7. Concurrency Big data is seated on heavy data workload; high concurrency is important. MySQL 8 is extending its maximum CPU core support in every new release, optimizing concurrency based on your requirements and hardware resources should be taken care of. 8. Hidden workloads Do not miss out factors that run in the background, such as reporting for big data analytics, backups, and on-the-fly operations while you are benchmarking. The impact of such hidden workloads or obsolete benchmarking workloads can make your days (and nights) Miserable. 9. Nerves of your query Oops! Did we miss the optimizer? Not yet. An optimizer is a powerful tool that will read the nerves of your query and provide recommendations. It's a tool that I use before making changes to a query in production. It's a savior when you have complex queries to be optimized. These are a few areas that we should look out for. Let's now look at a few benchmarks that we did on MySQL 8 and compare them with the ones on MySQL 5.7. 10. Benchmarks To start with, let's fetch all the column names from all the InnoDB tables. The following is the query that we executed: SELECT t.table_schema, t.table_name, c.column_name FROM information_schema.tables t, information_schema.columns c WHERE t.table_schema = c.table_schema AND t.table_name = c.table_name AND t.engine='InnoDB'; The following figure shows how MySQL 8 performed a thousand times faster when having four instances: Following this, we also performed a benchmark to find static table metadata. The following is the query that we executed: SELECT TABLE_SCHEMA, TABLE_NAME, TABLE_TYPE, ENGINE, ROW_FORMAT FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA LIKE 'chintan%'; The following figure shows how MySQL 8 performed around 30 times faster than MySQL 5.7:   It made us eager to go into a bit more detail. So, we thought of doing one last test to find dynamic table metadata. The following is the query that we executed: SELECT TABLE_ROWS FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA LIKE 'chintan%'; The following figure shows how MySQL 8 performed around 30 times faster than MySQL 5.7: MySQL 8.0 brings enormous performance improvement to the table. Scaling from one to million tables, is a need for big data requirements, which is now achievable. We look forward to more benchmarks being officially released once MySQL 8 is available for general purpose. If you found this post useful, make sure to check out the book MySQL 8 Administrator’s Guide for more tips and tricks to manage MySQL 8 effectively. MySQL 8.0 is generally available with added features New updates to Microsoft Azure services for SQL Server, MySQL, and PostgreSQL  

TensorFlow 2 is a rich development ecosystem composed of two main parts: Training and Serving. Training consists of a set of libraries for dealing with datasets (tf.data), a set of libraries for building models, including high-level libraries (tf.Keras and Estimators), low-level libraries (tf.*), and a collection of pretrained models (tf.Hub). Training can happen on CPUs, GPUs, and TPUs via distribution strategies and the result can be saved using the appropriate libraries.  This article is an excerpt from the book, Deep Learning with TensorFlow 2 and Keras, Second Edition by Antonio Gulli, Amita Kapoor, and Sujit Pal. This book teaches deep learning techniques alongside TensorFlow (TF) and Keras. In this article, we’ll review the addition of the powerful new feature, distributed training, in TensorFlow 2.x.  One very useful addition to TensorFlow 2.x is the possibility to train models using distributed GPUs, multiple machines, and TPUs in a very simple way with very few additional lines of code. tf.distribute.Strategy is the TensorFlow API used in this case and it supports both tf.keras and tf.estimator APIs and eager execution. You can switch between GPUs, TPUs, and multiple machines by just changing the strategy instance. Strategies can be synchronous, where all workers train over different slices of input data in a form of sync data parallel computation, or asynchronous, where updates from the optimizers are not happening in sync. All strategies require that data is loaded in batches via the tf.data.Dataset api.  Note that the distributed training support is still experimental. A roadmap is given in Figure 1:  Figure 1: Distributed training support fr different strategies and APIs  Let’s discuss in detail all the different strategies reported in Figure 1.  Multiple GPUs  TensorFlow 2.x can utilize multiple GPUs. If we want to have synchronous distributed training on multiple GPUs on one machine, there are two things that we need to do: (1) We need to load the data in a way that will be distributed into the GPUs, and (2) We need to distribute some computations into the GPUs too:  In order to load our data in a way that can be distributed into the GPUs, we simply need tf.data.Dataset (which has already been discussed in the previous paragraphs). If we do not have a tf.data.Dataset but we have a normal tensor, then we can easily convert the latter into the former using tf.data.Dataset.from_tensors_slices(). This will take a tensor in memory and return a source dataset, the elements of which are slices of the given tensor. In our toy example, we use NumPy to generate training data x and labels y, and we transform it into tf.data.Dataset with tf.data.Dataset.from_tensor_slices(). Then we apply a shuffle to avoid bias in training across GPUs and then generate SIZE_BATCHES batches:  import tensorflow as tf import numpy as np from tensorflow import keras N_TRAIN_EXAMPLES = 1024*1024 N_FEATURES = 10 SIZE_BATCHES = 256  # 10 random floats in the half-open interval [0.0, 1.0). x = np.random.random((N_TRAIN_EXAMPLES, N_FEATURES)) y = np.random.randint(2, size=(N_TRAIN_EXAMPLES, 1)) x = tf.dtypes.cast(x, tf.float32) print (x) dataset = tf.data.Dataset.from_tensor_slices((x, y)) dataset = dataset.shuffle(buffer_size=N_TRAIN_EXAMPLES).batch(SIZE_BATCHES) In order to distribute some computations to GPUs, we instantiate a distribution = tf.distribute.MirroredStrategy() object, which supports synchronous distributed training on multiple GPUs on one machine. Then, we move the creation and compilation of the Keras model inside the strategy.scope(). Note that each variable in the model is mirrored across all the replicas. Let’s see it in our toy example: # this is the distribution strategy distribution = tf.distribute.MirroredStrategy() # this piece of code is distributed to multiple GPUs with distribution.scope(): model = tf.keras.Sequential()   model.add(tf.keras.layers.Dense(16, activation=‘relu’, input_shape=(N_FEATURES,)))   model.add(tf.keras.layers.Dense(1, activation=‘sigmoid’))   optimizer = tf.keras.optimizers.SGD(0.2)   model.compile(loss=‘binary_crossentropy’, optimizer=optimizer) model.summary()  # Optimize in the usual way but in reality you are using GPUs. model.fit(dataset, epochs=5, steps_per_epoch=10)  Note that each batch of the given input is divided equally among the multiple GPUs. For instance, if using MirroredStrategy() with two GPUs, each batch of size 256 will be divided among the two GPUs, with each of them receiving 128 input examples for each step. In addition, note that each GPU will optimize on the received batches and the TensorFlow backend will combine all these independent optimizations on our behalf. In short, using multiple GPUs is very easy and requires minimal changes to the tf.Keras code used for a single server.  MultiWorkerMirroredStrategy  This strategy implements synchronous distributed training across multiple workers, each one with potentially multiple GPUs. As of September 2019 the strategy works only with Estimators and it has experimental support for tf.Keras. This strategy should be used if you are aiming at scaling beyond a single machine with high performance. Data must be loaded with tf.Dataset and shared across workers so that each worker can read a unique subset.  TPUStrategy  This strategy implements synchronous distributed training on TPUs. TPUs are Google’s specialized ASICs chips designed to significantly accelerate machine learning workloads in a way often more efficient than GPUs. According to this public information (https://github.com/tensorflow/tensorflow/issues/24412):  “the gist is that we intend to announce support for TPUStrategy alongside Tensorflow 2.1. Tensorflow 2.0 will work under limited use-cases but has many improvements (bug fixes, performance improvements) that we’re including in Tensorflow 2.1, so we don’t consider it ready yet.”  ParameterServerStrategy  This strategy implements either multi-GPU synchronous local training or asynchronous multi-machine training. For local training on one machine, the variables of the models are placed on the CPU and operations are replicated across all local GPUs. For multi-machine training, some machines are designated as workers and some as parameter servers with the variables of the model placed on parameter servers. Computation is replicated across all GPUs of all workers. Multiple workers can be set up with the environment variable TF_CONFIG as in the following example:  os.environ[“TF_CONFIG”] = json.dumps({    “cluster”: {        “worker”: [“host1:port”, “host2:port”, “host3:port”],         “ps”: [“host4:port”, “host5:port”]    },    “task”: {“type”: “worker”, “index”: 1} })  In this article, we have seen how it is possible to train models using distributed GPUs, multiple machines, and TPUs in a very simple way with very few additional lines of code. Learn how to build machine and deep learning systems with the newly released TensorFlow 2 and Keras for the lab, production, and mobile devices with Deep Learning with TensorFlow 2 and Keras, Second Edition by Antonio Gulli, Amita Kapoor and Sujit Pal.  About the Authors  Antonio Gulli is a software executive and business leader with a passion for establishing and managing global technological talent, innovation, and execution. He is an expert in search engines, online services, machine learning, information retrieval, analytics, and cloud computing.   Amita Kapoor is an Associate Professor in the Department of Electronics, SRCASW, University of Delhi and has been actively teaching neural networks and artificial intelligence for the last 20 years. She is an active member of ACM, AAAI, IEEE, and INNS. She has co-authored two books.   Sujit Pal is a technology research director at Elsevier Labs, working on building intelligent systems around research content and metadata. His primary interests are information retrieval, ontologies, natural language processing, machine learning, and distributed processing. He is currently working on image classification and similarity using deep learning models. He writes about technology on his blog at Salmon Run. 

Last week, a team of researchers from Stanford University, Max Planck Institute for Informatics, Princeton University and Adobe Research published a paper titled “Text-based Editing of Talking-head Video”. This paper proposes a method to edit a talking-head video based on its transcript to produce a realistic output video, in which the dialogue of the speaker has been modified. Basically, the editor modifies a video using a text transcript, to add new words, delete unwanted ones or completely rearrange the pieces by dragging and dropping. This video will maintain a seamless audio-visual flow, without any jump cuts and will look almost flawless to the untrained eye. The researchers want this kind of text-based editing approach to lay the foundation for better editing tools, in post production of movies and television. Actors often botch small bits of performance or leave out a critical word. This algorithm can help video editors fix that, which has until now involves expensive reshoots. It can also help in easy adaptation of audio-visual video content to specific target audiences. The tool supports three types of edit operations- add new words, rearrange existing words, delete existing words. Ohad Fried, a researcher in the paper says that “This technology is really about better storytelling. Instructional videos might be fine-tuned to different languages or cultural backgrounds, for instance, or children’s stories could be adapted to different ages.” https://youtu.be/0ybLCfVeFL4 How does the application work? The method uses an input talking-head video and a transcript to perform text-based editing. The first step is to align phonemes to the input audio and track each input frame to construct a parametric head model. Next, a 3D parametric face model with each frame of the input talking-head video is registered. This helps in selectively blending different aspects of the face. Then, a background sequence is selected and is used for pose data and background pixels. The background sequence allows editors to edit challenging videos with hair movement and slight camera motion. As Facial expressions are an important parameter, the researchers have tried to preserve the retrieved expression parameters as much as possible, by smoothing out the transition between them. This provides an output of edited parameter sequence which describes the new desired facial motion and a corresponding retimed background video clip. This is forwarded to a ‘neural face rendering’ approach. This step changes the facial motion of the retimed background video to match the parameter sequence. Thus the rendering procedure produces photo-realistic video frames of the subject, appearing to speak the new phrase.These localized edits seamlessly blends into the original video, producing an edited result. Lastly to add the audio, the resulted video is retimed to match the recording at the level of phones. The researchers have used the performers own voice in all their synthesis results. Image Source: Text-based Editing of Talking-head Video The researchers have tested the system with a series of complex edits including adding, removing and changing words, as well as translations to different languages. When the application was tried in a crowd-sourced study with 138 participants, the edits were rated as “real”, almost 60% of the time. Fried said that “The visual quality is such that it is very close to the original, but there’s plenty of room for improvement.” Ethical considerations: Erosion of truth, confusion and defamation Even though the application is quite useful for video editors and producers, it raises important and valid concerns about its potential for misuse. The researchers have also agreed that such a technology might be used for illicit purposes. “We acknowledge that bad actors might use such technologies to falsify personal statements and slander prominent individuals. We are concerned about such deception and misuse.” They have recommended certain precautions to be taken to avoid deception and misuse such as using watermarking. “The fact that the video is synthesized may be obvious by context, directly stated in the video or signaled via watermarking. We also believe that it is essential to obtain permission from the performers for any alteration before sharing a resulting video with a broad audience.” They urge the community to continue to develop forensics, fingerprinting and verification techniques to identify manipulated video. They also support the creation of appropriate regulations and laws that would balance the risks of misuse of these tools against the importance of creative, consensual use cases. The public however remain dubious pointing out valid arguments on why the ‘Ethical Concerns’ talked about in the paper, fail. A user on Hacker News comments, “The "Ethical concerns" section in the article feels like a punt. The author quoting "this technology is really about better storytelling" is aspirational -- the technology's story will be written by those who use it, and you can bet people will use this maliciously.” https://twitter.com/glenngabe/status/1136667296980701185 Another user feels that such kind of technology will only result in “slow erosion of video evidence being trustworthy”. Others have pointed out how the kind of transformation mentioned in the paper, does not come under the broad category of ‘video-editing’ ‘We need more words to describe this new landscape’ https://twitter.com/BrianRoemmele/status/1136710962348617728 Another common argument is that the algorithm can be used to generate terrifyingly real Deepfake videos. A Shallow Fake video was Nancy Pelosi’s altered video, which circulated recently, that made it appear she was slurring her words by slowing down the video. Facebook was criticized for not acting faster to slow the video’s spread. Not just altering speeches of politicians, altered videos like these can also, for instance, be used to create fake emergency alerts, or disrupt elections by dropping a fake video of one of the candidates before voting starts. There is also the issue of defaming someone on a personal capacity. Sam Gregory, Program Director at Witness, tweets that one of the main steps in ensuring effective use of such tools would be to “ensure that any commercialization of synthetic media tools has equal $ invested in detection/safeguards as in detection.; and to have a grounded conversation on trade-offs in mitigation”. He has also listed more interesting recommendations. https://twitter.com/SamGregory/status/1136964998864015361 For more details, we recommend you to read the research paper. OpenAI researchers have developed Sparse Transformers, a neural network which can predict what comes next in a sequence ‘Facial Recognition technology is faulty, racist, biased, abusive to civil rights; act now to restrict misuse’ say experts to House Oversight and Reform Committee Now there’s a CycleGAN to visualize the effects of climate change. But is this enough to mobilize action?

Dive deeper into the world of AI innovation and stay ahead of the AI curve! Subscribe to our AI_Distilled newsletter for the latest insights. Don't miss out – sign up today!IntroductionIn an era dominated by the rise of artificial intelligence, the power and promise of Large Language Models (LLMs) stand distinct. These colossal architectures, designed to understand and generate human-like text, have revolutionized the realm of natural language processing. However, with great power comes great responsibility – the onus of managing, deploying, and refining these models in real-world scenarios. This article delves into the world of Large Language Model Operations (LLMOps), an emerging field that bridges the gap between the potential of LLMs and their practical application.BackgroundThe last decade has seen a significant evolution in language models, with models growing in size and capability. Starting with smaller models like Word2Vec and LSTM, we've advanced to behemoths like GPT-3, BERT, and T5.  With that said, as these models grew in size and complexity, so did their operational challenges. Deploying, maintaining, and updating these models requires substantial computational resources, expertise, and effective management strategies.MLOps vs LLMOpsIf you've ventured into the realm of machine learning, you've undoubtedly come across the term MLOps. MLOps, or Machine Learning Operations, encapsulates best practices and methodologies for deploying and maintaining machine learning models throughout their lifecycle. It caters to the wide spectrum of models that fall under the machine learning umbrella.On the other hand, with the growth of vast and intricate language models, a more specialized operational domain has emerged: LLMOps. While both MLOps and LLMOps share foundational principles, the latter specifically zeros in on the challenges and nuances of deploying and managing large-scale language models. Given the colossal size, data-intensive nature, and unique architecture of these models, LLMOps brings to the fore bespoke strategies and solutions that are fine-tuned to ensure the efficiency, efficacy, and sustainability of such linguistic powerhouses in real-world scenarios.Core Concepts of LLMOpsLarge Language Models Operations (LLMOps) focuses on the management, deployment, and optimization of large language models (LLMs). One of its foundational concepts is model deployment, emphasizing scalability to handle varied loads, reducing latency for real-time responses, and maintaining version control. As these LLMs demand significant computational resources, efficient resource management becomes pivotal. This includes the use of optimized hardware like GPUs and TPUs, effective memory optimization strategies, and techniques to manage computational costs.Continuous learning and updating, another core concept, revolve around fine-tuning models with new data, avoiding the pitfall of 'catastrophic forgetting', and effectively managing data streams for updates. Parallelly, LLMOps emphasizes the importance of continuous monitoring for performance, bias, fairness, and iterative feedback loops for model improvement. To cater to the vastness of LLMs, model compression techniques like pruning, quantization, and knowledge distillation become crucial.How do LLMOps workPre-training Model DevelopmentLarge Language Models typically start their journey through a process known as pre-training. This involves training the model on vast amounts of text data. The objective during this phase is to capture a broad understanding of language, learning from billions of sentences and paragraphs. This foundational knowledge helps the model grasp grammar, vocabulary, factual information, and even some level of reasoning.This massive-scale training is what makes them "large" and gives them a broad understanding of language. Optimization & CompressionModels trained to this extent are often so large that they become impractical for daily tasks.To make these models more manageable without compromising much on performance, techniques like model pruning, quantization, and knowledge distillation are employed.Model Pruning: After training, pruning is typically the first optimization step. This begins with trimming model weights and may advance to more intensive methods like neuron or channel pruning.Quantization: Following pruning, the model's weights, and potentially its activations, are streamlined. Though weight quantization is generally a post-training process, for deeper reductions, such as very low-bit quantization, one might adopt quantization-aware training from the beginning.Additional recommendations are:Optimizing the model specifically for the intended hardware can elevate its performance. Before initiating training, selecting inherently efficient architectures with fewer parameters is beneficial. Approaches that adopt parameter sharing or tensor factorization prove advantageous. For those planning to train a new model or fine-tune an existing one with an emphasis on sparsity, starting with sparse training is a prudent approach.Deployment Infrastructure After training and compressing our LLM, we will be using technologies like Docker and Kubernetes to deploy models scalably and consistently. This approach allows us to flexibly scale using as many pods as needed. Concluding the deployment process, we'll implement edge deployment strategies. This positions our models nearer to the end devices, proving crucial for applications that demand real-time responses.Continuous Monitoring & FeedbackThe process starts with the Active model in production. As it interacts with users and as language evolves, it can become less accurate, leading to the phase where the Model becomes stale as time passes.To address this, feedback and interactions from users are captured, forming a vast range of new data. Using this data, adjustments are made, resulting in a New fine-tuned model.As user interactions continue and the language landscape shifts, the current model is replaced with the new model. This iterative cycle of deployment, feedback, refinement, and replacement ensures the model always stays relevant and effective.Importance and Benefits of LLMOpsMuch like the operational paradigms of AIOps and MLOps, LLMOps brings a wealth of benefits to the table when managing Large Language Models.MaintenanceAs LLMs are computationally intensive. LLMOps streamlines their deployment, ensuring they run smoothly and responsively in real-time applications. This involves optimizing infrastructure, managing resources effectively, and ensuring that models can handle a wide variety of queries without hiccups.Consider the significant investment of effort, time, and resources required to maintain Large Language Models like Chat GPT, especially given its vast user base.Continuous ImprovementLLMOps emphasizes continuous learning, allowing LLMs to be updated with fresh data. This ensures that models remain relevant, accurate, and effective, adapting to the evolving nature of language and user needs.Building on the foundation of GPT-3, the newer GPT-4 model brings enhanced capabilities. Furthermore, while ChatGPT was previously trained on data up to 2021, it has now been updated to encompass information through 2022.It's important to recognize that constructing and sustaining large language models is an intricate endeavor, necessitating meticulous attention and planning.ConclusionThe ascent of Large Language Models marks a transformative phase in the evolution of machine learning. But it's not just about building them; it's about harnessing their power efficiently, ethically, and sustainably. LLMOps emerge as the linchpin, ensuring that these models not only serve their purpose but also evolve with the ever-changing dynamics of language and user needs. As we continue to innovate, the principles of LLMOps will undoubtedly play a pivotal role in shaping the future of language models and their place in our digital world.Author BioMostafa Ibrahim is a dedicated software engineer based in London, where he works in the dynamic field of Fintech. His professional journey is driven by a passion for cutting-edge technologies, particularly in the realms of machine learning and bioinformatics. When he's not immersed in coding or data analysis, Mostafa loves to travel.Medium

IntroductionIn the ever-evolving landscape of artificial intelligence, a groundbreaking collaboration has emerged between Wolfram Alpha and ChatGPT, giving birth to an extraordinary plugin: the AI Advantage. This partnership bridges the gap between ChatGPT's proficiency in natural language processing and Wolfram Alpha's computational prowess. The result? A fusion that unlocks an array of new possibilities, revolutionizing the way we interact with AI. In this hands-on tutorial, we're embarking on a journey to explore the power of the Wolfram Alpha API, demonstrate its integration with Python and ChatGPT, and empower you to tap into this dynamic duo for tasks ranging from complex calculations to real-time data retrieval.Understanding Wolfram Alpha APIImagine having an intelligent assistant at your fingertips, capable of not only understanding your questions but also providing detailed computational insights. That's where Wolfram Alpha shines. It's more than just a search engine; it's a computational knowledge engine. Whether you need to solve a math problem, retrieve real-time data, or generate visual content, Wolfram Alpha has you covered. Its unique ability to compute answers based on structured data sets it apart from traditional search engines.So, how can you tap into this treasure trove of computational knowledge? Enter the Wolfram Alpha API. This API exposes Wolfram Alpha's capabilities for developers to harness in their applications. Whether you're building a chatbot, a data analysis tool, or an educational resource, the Wolfram Alpha API can provide you with instant access to accurate and in-depth information. The API supports a wide range of queries, from straightforward calculations to complex data retrievals, making it a versatile tool for various use cases.Integrating Wolfram Alpha API with ChatGPTChatGPT's strength lies in its ability to understand and generate human-like text based on input. However, when it comes to intricate calculations or pulling real-time data, it benefits from a partner like Wolfram Alpha. By integrating the two, you create a dynamic synergy where ChatGPT can effortlessly tap into Wolfram Alpha's computational engine to provide accurate and data-driven responses. This collaboration bridges the gap between language understanding and computation, resulting in a well-rounded AI interaction.Before we dive into the technical implementation, let's get you set up to take advantage of the Wolfram Alpha plugin for ChatGPT. First, ensure you have access to ChatGPT+. To enable the Wolfram plugin, follow these steps:Open the ChatGPT interface.Navigate to "Settings."Look for the "Beta Features" section.Enable "Plugins" under the GPT-4 options.Once "Plugins" is enabled, locate and activate the Wolfram plugin.With the plugin enabled you're ready to harness the combined capabilities of ChatGPT and Wolfram Alpha API, making your AI interactions more robust and informative.In the next sections, we'll dive into practical applications and walk you through implementing the integration using Python and ChatGPT.Practical Applications with Code ExamplesLet's start by exploring how the Wolfram Alpha API can assist with complex mathematical tasks. Below are code examples that demonstrate the integration between ChatGPT and Wolfram Alpha to solve intricate math problems. In these scenarios, ChatGPT serves as the bridge between you and Wolfram Alpha, seamlessly delivering accurate solutions.Before diving into the code implementation, let's ensure your environment is ready to go. Follow these steps to set up the necessary components:Install the required packages: Make sure you have the necessary Python packages installed. You can use pip to install them:pip install langchain openai wolframalphaNow, let's walk through implementing the code example you provided earlier. This code integrates the Wolfram Alpha API with ChatGPT to provide accurate and informative responses:Wolfram Alpha can solve simple arithmetic queries:# User input question = "Solve for x: 2x + 5 = 15" # Let ChatGPT interact with Wolfram Alpha response = agent_chain.run(input=question) # Extracting and displaying the result from the response result = response['text'] print("Solution:", result)Or mode complex ones like calculating integrals:# User input question = "Calculate the integral of x^2 from 0 to 5" # Let ChatGPT interact with Wolfram Alpha response = agent_chain.run(input=question) # Extracting and displaying the result from the response print("Integral:", response) Real-time Data RetrievalIncorporating real-time data into conversations can greatly enhance the value of AI interactions. Here are code examples showcasing how to retrieve up-to-date information using the Serper API and integrate it seamlessly into the conversation:# User input question = "What's the current exchange rate between USD and EUR?" # Let ChatGPT interact with Wolfram Alpha response = agent_chain.run(input=question) # Extracting and displaying the result from the response print("Exchange Rate:", response)We can also ask for the current weather forecast:# User input question = "What's the weather forecast for London tomorrow?" # Let ChatGPT interact with Wolfram Alpha response = agent_chain.run(input=question) # Extracting and displaying the result from the response print("Weather Forecast:", response)Now we can put everything together into a single block including all the required library imports and use both real time data with Serper and use the reasoning skills of Wolfram Alpha.# Import required libraries from langchain.agents import load_tools, initialize_agent from langchain.llms import OpenAI from langchain.memory import ConversationBufferMemory from langchain.chat_models import ChatOpenAI # Set environment variables import os os.environ['OPENAI_API_KEY'] = 'your-key' os.environ['WOLFRAM_ALPHA_APPID'] = 'your-key' os.environ["SERPER_API_KEY"] = 'your-key' # Initialize the ChatGPT model llm = ChatOpenAI(temperature=0, model="gpt-3.5-turbo") # Load tools and set up memory tools = load_tools(["google-serper", "wolfram-alpha"], llm=llm) memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True) # Initialize the agent agent_chain = initialize_agent(tools, llm, handle_parsing_errors=True, verbose=True, memory=memory) # Interact with the agent response_weather = agent_chain.run(input="what is the weather in Amsterdam right now in celcius? Don't make assumptions.") response_flight = agent_chain.run(input="What's a good price for a flight from JFK to AMS this weekend? Express the price in Euros. Don't make assumptions.")ConclusionIn this tutorial, we've delved into the exciting realm of integrating the Wolfram Alpha API with Python and ChatGPT. We've explored how this collaboration empowers you to tackle complex mathematical tasks and retrieve real-time data seamlessly. By harnessing the capabilities of both Wolfram Alpha and ChatGPT, you've unlocked a powerful synergy that's capable of transforming your AI interactions. As you continue to explore and experiment with this integration, you'll discover new ways to enhance your interactions and leverage the strengths of each tool. So, why wait? Start your journey toward more informative and engaging AI interactions today.Author BioAlan Bernardo Palacio is a data scientist and an engineer with vast experience in different engineering fields. His focus has been the development and application of state-of-the-art data products and algorithms in several industries. He has worked for companies such as Ernst and Young, Globant, and now holds a data engineer position at Ebiquity Media helping the company to create a scalable data pipeline. Alan graduated with a Mechanical Engineering degree from the National University of Tucuman in 2015, participated as the founder of startups, and later on earned a Master's degree from the faculty of Mathematics in the Autonomous University of Barcelona in 2017. Originally from Argentina, he now works and resides in the Netherlands.LinkedIn 

Data science is a field that’s complex and diverse. If you’re trying to learn data science and become a data scientist it can be easy to fall down a rabbit hole of machine learning or data processing. To a certain extent, that’s good. To be an effective data scientist you need to be curious. You need to be prepared to take on a range of different tasks and challenges. But that’s not always that efficient: if you want to learn quickly and effectively, you need a clear structure - a curriculum - that you can follow. This post will show you what you need to learn and how to go about it. Statistics Statistics is arguably the cornerstone of data science. Nate Silver called data scientists “sexed up statisticians”, a comment that was perhaps unfair but still nevertheless contains a kernel of truth in it: that data scientists are always working in the domain of statistics. Once you understand this everything else you need to learn will follow easily. Machine learning, data manipulation, data visualization - these are all ultimately technological methods for performing statistical analysis really well. Best Packt books and videos content for learning statistics Statistics for Data Science R Statistics Cookbook Statistical Methods and Applied Mathematics in Data Science [Video] Before you go any deeper into data science, it’s critical that you gain a solid foundation in statistics. Data mining and wrangling This is an important element of data science that often gets overlooked with all the hype about machine learning. However, without effective data collection and cleaning, all your efforts elsewhere are going to be pointless at best. At worst they might even be misleading or problematic. Sometimes called data manipulation or data munging, it's really all about managing and cleaning data from different sources so it can be used for analytics projects. To do it well you need to have a clear sense of where you want to get to - do you need to restructure the data? Sort or remove certain parts of a data set? Once you understand this, it’s much easier to wrangle data effectively. Data mining and wrangling tools There are a number of different tools you can use for data wrangling. Python and R are the two key programming languages, and both have some useful tools for data mining and manipulation. Python in particular has a great range of tools for data mining and wrangling, such as pandas and NLTK (Natural Language Toolkit), but that isn’t to say R isn’t powerful in this domain. Other tools are available too - Weka and Apache Mahout, for example, are popular. Weka is written in Java so is a good option if you have experience with that programming language, while Mahout integrates well with the Hadoop ecosystem. Data mining and data wrangling books and videos If you need to learn data mining, wrangling and manipulation, Packt has a range of products. Here are some of the best: Data Wrangling with R Data Wrangling with Python Python Data Mining Quick Start Guide Machine Learning for Data Mining Machine learning and artificial intelligence Although Machine learning and artificial intelligence are huge trends in their own right, they are nevertheless closely aligned with data science. Indeed, you might even say that their prominence today has grown out of the excitement around data science that we first we witnessed just under a decade ago. It’s a data scientist’s job to use machine learning and artificial intelligence in a way that can drive business value. That could, for example, be to recommend products or services to customers, perhaps to gain a better understanding into existing products, or even to better manage strategic and financial risks through predictive modelling. So, while we can see machine learning in a massive range of digital products and platforms - all of which require smart development and design - for it to work successfully, it needs to be supported by a capable and creative data scientist. Machine learning and artificial intelligence books for data scientists Machine Learning Algorithms Machine Learning with R - Third Edition Machine Learning with Apache Spark Quick Start Guide Machine Learning with TensorFlow 1.x Keras Deep Learning Cookbook Data visualization A talented data scientist isn’t just a great statistician and engineer, they’re also a great communicator. This means so-called soft skills are highly valuable - the ability to communicate insights and ideas with key stakeholders is essential. But great communication isn’t just about soft skills, it’s also about data visualization. Data visualization is, at a fundamental level, about organizing and presenting data in a way that tells a story, clarifies a problem, or illustrates a solution. It’s essential that you don’t overlook this step. Indeed, spending time learning about effective data visualization can also help you to develop your soft skills. The principles behind storytelling and communication through visualization are, in truth, exactly the same when applied to other scenarios. Data visualization tools There are a huge range of data visualization tools available. As with machine learning, understanding the differences between them and working out what solution will work for you is actually an important part of the learning process. For that reason, don’t be afraid to spend a little bit of time with a range of data visualization tools. Many of the most popular data visualization tools are paid for products. Perhaps the best known of these is Tableau (which, incidentally was bought by Salesforce earlier this year). Tableau and its competitors are very user friendly, which means the barrier to entry is pretty low. They allow you to create some pretty sophisticated data visualizations fairly easily. However, sticking to these tools is not only expensive, it can also limit your abilities. We’d recommend trying a number of different data visualization tools, such as Seabor, D3.js, Matplotlib, and ggplot2. Data visualization books and videos for data scientists Applied Data Visualization with R and ggplot2 Tableau 2019.1 for Data Scientists [Video] D3.js Data Visualization Projects [Video] Tableau in 7 Steps [Video] Data Visualization with Python If you want to learn data science, just get started! As we've seen, data science requires a number of very different skills and takes in a huge breadth of tools. That means that if you're going to be a data scientist, you need to be prepared to commit to learning forver: you're never going to reach a point where you know everything. While that might sound intimidating, it's important to have confidence. With a sense of direction and purpose, and a learning structure that works for you, it's possible to develop and build your data science capabilities in a way that could unlock new opportunities and act as the basis for some really exciting projects.

According to a complaint filed in a lawsuit yesterday, Google paid $135 million in total as exit packages to top two senior execs, namely Andy Rubin (creator of Android) and Amit Singhal (former senior VP of Google search) after they were accused of sexual misconduct in the company. The lawsuit was filed by an Alphabet shareholder, James Martin, in the Santa Clara, California Court. Google also confirmed paying the exit packages to senior execs to The Verge, yesterday. Speaking of the lawsuit, the complaint is against certain directors and officers of Alphabet, Google’s parent company, for their active and direct participation in “multi-year scheme” to hide sexual harassment and discrimination at Alphabet. It also states that the misconduct by these directors has caused severe financial and reputational damage to Alphabet. The exit packages for Rubin and Singhal were approved by the Leadership Development and Compensation Committee (LLDC). The news of Google paying high exit packages to its top execs first came to light last October, after the New York Times released a report on Google, stating that the firm paid $90 million to Rubin and $15 million to Singhal. Rubin had previously also received an offer for a $150 million stock grant, which he then further use to negotiate the $90 million in severance pay, even though he should have been fired for cause without any pay, states the lawsuit. To protest against the handling of sexual misconduct within Google, more than 20,000 Google employees along with vendors, and contractors, temps, organized Google “walkout for real change” and walked out of their offices in November 2018. Googlers also launched an industry-wide awareness campaign to fight against forced arbitration in January, where they shared information about arbitration on their Twitter and Instagram accounts throughout the day.   Last year in November, Google ended its forced arbitration ( a move that was soon followed by Facebook) for its employees (excluding temps, vendors, etc) and only in the case of sexual harassment. This led to contractors writing an open letter on Medium to Sundar Pichai, CEO, Google, in December, demanding him to address their demands of better conditions and equal benefits for contractors. In response to the Google walkout and the growing public pressure, Google finally decided to end its forced arbitration policy for all employees (including contractors) and for all kinds of discrimination within Google, last month. The changes will go into effect for all the Google employees starting March 21st, 2019. Yesterday, the Google walkout for real change group tweeted condemning the multi-million dollar payouts and has asked people to use the hashtag #Googlepayoutsforall to highlight other better ways that money could have been used. https://twitter.com/GoogleWalkout/status/1105450565193121792 “The conduct of Rubin and other executives was disgusting, illegal, immoral, degrading to women and contrary to every principle that Google claims it abides by”, reads the lawsuit. James Martin also filed a lawsuit against Alphabet’s board members, Larry Page, Sergey Brin, and Eric Schmidt earlier this year in January for covering up the sexual harassment allegations against the former top execs at Google. Martin had sued Alphabet for breaching its fiduciary duty to shareholders, unjust enrichment, abuse of power, and corporate waste. “The directors’ wrongful conduct allowed illegal conduct to proliferate and continue. As such, members of the Alphabet’s board were knowing direct enables of sexual harassment and discrimination”, reads the lawsuit. It also states that the board members not only violated the California and federal law but it also violated the ethical standards and guidelines set by Alphabet. Public reaction to the news is largely negative with people condemning Google’s handling of sexual misconduct: https://twitter.com/awesome/status/1105295877487263744 https://twitter.com/justkelly_ok/status/1105456081663225856 https://twitter.com/justkelly_ok/status/1105457965790707713 https://twitter.com/conradwt/status/1105386882135875584 https://twitter.com/mer__edith/status/1105464808831361025 For more information, check out the official lawsuit here. Recode Decode #GoogleWalkout interview shows why data and evidence don’t always lead to right decisions in even the world’s most data-driven company Liz Fong Jones, prominent ex-Googler shares her experience at Google and ‘grave concerns’ for the company Google’s pay equity analysis finds men, not women, are underpaid; critics call out design flaws in the analysis

The Jupyter notebook is an interactive notebook allowing you to write documents with embedded code, and execute this code on the fly. It was originally developed as a part of the Ipython project, and could only be used for Python code at that time. Nowadays, the Jupyter notebook integrates multiple languages, such as R, Julia, Haskell and much more - the notebook supports about 50 languages. One of the best features of the notebook is to be able to mix code and markdown with embedded HTML/CSS. It allows an easy creation of beautiful interactive documents, such as documentation examples. It can also help with the writing of full documentation using its export mechanism to HTML, RST (ReStructured Text), markdown and PDF. Interactive documentation examples When writing library documentation, a lot of time should be dedicated to writing examples for each part of the library. However, those examples are quite often static code, each part being explained through comments. To improve the writing of those examples, a solution could be using a Jupyter notebook, which can be downloaded and played with by anyone reading your library documentation. Solutions also exist to have the notebook running directly on your website, as seen on the Jupyter website, where you can try the notebooks. This will not be explained in this post, but the notebook was designed on a server-client pattern, making this easy to get running. Using the notebook cells capabilities, you can separate each part of your code, or each example, and describe it nicely and properly outside the code cell, improving readability. From the Jupyter Python notebook example, we see what the following code does, execute it (and even get graphics back directly on the notebook!). Here is an example of a Jupyter notebook, for the Python language, with matplotlib integration: Even more than that, instead of just having your example code in a file, people downloading your notebook will directly get the full example documentation, giving them a huge advantage in understanding what the example is and what it does when opening the notebook again after six months. And they can just hit the run button, and tinker with your example to understand all its details, without having to go back and forth between the website and their code editor, saving them time. They will love you for that! Generate documentation from your notebooks Developing mainly in Python, I am used to the Sphinx library as a documentation generator. It can export your documentation to HTML from RST files, and scoops your code library to generate documentation from docstring, all with a single command, making it quite useful in the process of writing. As Jupyter notebooks can be exported to RST, why not use this mechanism to create your RST files with Jupyter, then generate your full documentation with Sphinx? To manually convert your notebook, you can click on File -> Download As -> reST. You will be prompted to download the file. That's it! Your notebook was exported. However, while this method is good for testing purposes, this will not be good for an automatic generation of documentation with sphinx. To be able to convert automatically, we are going to use a tool named nbconvert with which can do all the required conversions from the command line. To convert your notebook to RST, you just need to do the following $ jupyter nbconvert --to rst *your_notebook*.ipynb or to convert every notebook in the current folder: $ jupyter nbconvert --to rst *.ipynb Those commands can easily be integrated in a Makefile for your documentation, making the process of converting your notebooks completely invisible. If you want to keep your notebooks in a folder notebooks and your generated files in a folder rst, you can run assuming you have the following directory tree: Current working directory | |-rst/ |-notebooks/ |-notebook1.ipynb |-notebook2.ipynb |-... the following commands: $ cd rst $ jupyter nbconvert --to rst ../notebooks/*.ipynb This will convert all the notebooks in notebooks and place them in the rst folder. A Python API is also available if you want to generate your documentation from Python (Documentation). A lot more export options are available on the nbconvert documentation. You can create PDF, HTML or even slides, if you want to make a presentation based on a notebook, and can even pull a presentation from a remote location. Jupyter notebooks are very versatile documents, allowing interactive code exploration, export to a large number of formats, remote work, collaborative work and more. You can find more information on the official Jupyter website where you will also be able to try it. I mainly focused this post on the Python language, in which the IPython notebooks, ancestor of Jupyter were developed, but with the integration of more than 50 languages, it makes it a tool that every developer should be aware of, to create documentation, tutorials, or just to try code and keep notes at the same place. Dive deeper into the Jupyter Notebook by navigating your way around the dashboard and interface in our article. Read now! About the author Marin Gilles is a PhD student in Physics, in Dijon, France. A large part of his work is dedicated to physical simulations for which he developed his own simulation framework using Python, and contributed to open-source libraries such as Matplotlib or Ipython.

In this article by Bass Jobsen, author of the book Sass and Compass Designer's Cookbook you will learn the following topics: Installing Grunt Installing Grunt plugins Utilizing the Gruntfile.js file Adding a configuration definition for a plugin Adding the Sass compiler task (For more resources related to this topic, see here.) This article introduces you to the Grunt Task Runner and the features it offers to make your development workflow a delight. Grunt is a JavaScript Task Runner that is installed and managed via npm, the Node.js package manager. You will learn how to take advantage of its plugins to set up your own flexible and productive workflow, which will enable you to compile your Sass code. Although there are many applications available for compiling Sass, Grunt is a more flexible, versatile, and cross-platform tool that will allow you to automate many development tasks, including Sass compilation. It can not only automate the Sass compilation tasks, but also wrap any other mundane jobs, such as linting and minifying and cleaning your code, into tasks and run them automatically for you. By the end of this article, you will be comfortable using Grunt and its plugins to establish a flexible workflow when working with Sass. Using Grunt in your workflow is vital. You will then be shown how to combine Grunt's plugins to establish a workflow for compiling Sass in real time. Grunt becomes a tool to automate integration testing, deployments, builds, and development in which you can use. Finally, by understanding the automation process, you will also learn how to use alternative tools, such as Gulp. Gulp is a JavaScript task runner for node.js and relatively new in comparison to Grunt, so Grunt has more plugins and a wider community support. Currently, the Gulp community is growing fast. The biggest difference between Grunt and Gulp is that Gulp does not save intermediary files, but pipes these files' content in memory to the next stream. A stream enables you to pass some data through a function, which will modify the data and then pass the modified data to the next function. In many situations, Gulp requires less configuration settings, so some people find Gulp more intuitive and easier to learn. In this article, Grunt has been chosen to demonstrate how to run a task runner; this choice does not mean that you will have to prefer the usage of Grunt in your own project. Both the task runners can run all the tasks described in this article. Simply choose the task runner that suits you best. This recipe demonstrates shortly how to compile your Sass code with Gulp. In this article, you should enter your commands in the command prompt. Linux users should open a terminal, while Mac users should run Terminal.app and Window users should use the cmd command for command line usage. Installing Grunt Grunt is essentially a Node.js module; therefore, it requires Node.js to be installed. The goal of this recipe is to show you how to install Grunt on your system and set up your project. Getting ready Installing Grunt requires both Node.js and npm. Node.js is a platform built on Chrome's JavaScript runtime for easily building fast, scalable network applications, and npm is a package manager for Node.js. You can download the Node.js source code or a prebuilt installer for your platform at https://nodejs.org/en/download/. Notice that npm is bundled with node. Also, read the instructions at https://github.com/npm/npm#super-easy-install. How to do it... After installing Node.js and npm, installing Grunt is as simple as running a single command, regardless of the operating system that you are using. Just open the command line or the Terminal and execute the following command: npm install -g grunt-cli That's it! This command will install Grunt globally and make it accessible anywhere on your system. Run the grunt --version command in the command prompt in order to confirm that Grunt has been successfully installed. If the installation is successful, you should see the version of Grunt in the Terminal's output: grunt --version grunt-cli v0.1.11 After installing Grunt, the next step is to set it up for your project: Make a folder on your desktop and call it workflow. Then, navigate to it and run the npm init command to initialize the setup process: mkdir workflow && cd $_ && npm init Press Enter for all the questions and accept the defaults. You can change these settings later. This should create a file called package.json that will contain some information about the project and the project's dependencies. In order to add Grunt as a dependency, install the Grunt package as follows: npm install grunt --save-dev Now, if you look at the package.json file, you should see that Grunt is added to the list of dependencies: ..."devDependencies": {"grunt": "~0.4.5" } In addition, you should see an extra folder created. Called node_modules, it will contain Grunt and other modules that you will install later in this article. How it works... In the preceding section, you installed Grunt (grunt-cli) with the -g option. The -g option installs Grunt globally on your system. Global installation requires superuser or administrator rights on most systems. You need to run only the globally installed packages from the command line. Everything that you will use with the require() function in your programs should be installed locally in the root of your project. Local installation makes it possible to solve your project's specific dependencies. More information about global versus local installation of npm modules can be found at https://www.npmjs.org/doc/faq.html. There's more... Node package managers are available for a wide range of operation systems, including Windows, OSX, Linux, SunOS, and FreeBSD. A complete list of package managers can be found at https://github.com/joyent/node/wiki/Installing-Node.js-via-package-manager. Notice that these package managers are not maintained by the Node.js core team. Instead, each package manager has its own maintainer. See also The npm Registry is a public collection of packages of open source code for Node.js, frontend web apps, mobile apps, robots, routers, and countless other needs of the JavaScript community. You can find the npm Registry at https://www.npmjs.org/. Also, notice that you do not have to use Task Runners to create build chains. Keith Cirkel wrote about how to use npm as a build tool at http://blog.keithcirkel.co.uk/how-to-use-npm-as-a-build-tool/. Installing Grunt plugins Grunt plugins are the heart of Grunt. Every plugin serves a specific purpose and can also work together with other plugins. In order to use Grunt to set up your Sass workflow, you need to install several plugins. You can find more information about these plugins in this recipe's How it works... section. Getting ready Before you install the plugins, you should first create some basic files and folders for the project. You should install Grunt and create a package.json file for your project. Also, create an index.html file to inspect the results in your browser. Two empty folders should be created too. The scss folder contains your Sass code and the css folder contains the compiled CSS code. Navigate to the root of the project, repeat the steps from the Installing Grunt recipe of this article, and create some additional files and directories that you are going to work with throughout the article. In the end, you should end up with the following folder and file structure: How to do it... Grunt plugins are essentially Node.js modules that can be installed and added to the package.json file in the list of dependencies using npm. To do this, follow the ensuing steps: Navigate to the root of the project and run the following command, as described in the Installing Grunt recipe of this article: npm init Install the modules using npm, as follows: npm install grunt-contrib-sass load-grunt-tasks grunt-postcss --save-dev Notice the single space before the backslash in each line. For example, on the second line, grunt-contrib-sass , there is a space before the backslash at the end of the line. The space characters are necessary because they act as separators. The backslash at the end is used to continue the commands on the next line. The npm install command will download all the plugins and place them in the node_modules folder in addition to including them in the package.json file. The next step is to include these plugins in the Gruntfile.js file. How it works... Grunt plugins can be installed and added to the package.json file using the npm install command followed by the name of the plugins separated by a space, and the --save-dev flag: npm install nameOfPlugin1 nameOfPlugin2 --save-dev The --save-dev flag adds the plugin names and a tilde version range to the list of dependencies in the package.json file so that the next time you need to install the plugins, all you need to do is run the npm install command. This command looks for the package.json file in the directory from which it was called, and will automatically download all the specified plugins. This makes porting workflows very easy; all it takes is copying the package.json file and running the npm install command. Finally, the package.json file contains a JSON object with metadata. It is also worth explaining the long command that you have used to install the plugins in this recipe. This command installs the plugins that are continued on to the next line by the backslash. It is essentially equivalent to the following: npm install grunt-contrib-sass –-save-dev npm install load-grunt-tasks –-save-dev npm install grunt-postcss –-save-dev As you can see, it is very repetitive. However, both yield the same results; it is up to you to choose the one that you feel more comfortable with. The node_modules folder contains all the plugins that you install with npm. Every time you run npm install name-of-plugin, the plugin is downloaded and placed in the folder. If you need to port your workflow, you do not need to copy all the contents of the folder. In addition, if you are using a version control system, such as Git, you should add the node_modules folder to the .gitignore file so that the folder and its subdirectories are ignored. There's more... Each Grunt plugin also has its own metadata set in a package.json file, so plugins can have different dependencies. For instance, the grunt-contrib-sass plugin, as described in the Adding the Sass compiler task recipe, has set its dependencies as follows: "dependencies": {     "async": "^0.9.0",     "chalk": "^0.5.1",     "cross-spawn": "^0.2.3",     "dargs": "^4.0.0",     "which": "^1.0.5"   } Besides the dependencies described previously, this task also requires you to have Ruby and Sass installed. In the following list, you will find the plugins used in this article, followed by a brief description: load-grunt-tasks: This loads all the plugins listed in the package.json file grunt-contrib-sass: This compiles Sass files into CSS code grunt-postcss: This enables you to apply one or more postprocessors to your compiled CSS code CSS postprocessors enable you to change your CSS code after compilation. In addition to installing plugins, you can remove them as well. You can remove a plugin using the npm uninstall name-of-plugin command, where name-of-plugin is the name of the plugin that you wish to remove. For example, if a line in the list of dependencies of your package.json file contains grunt-concurrent": "~0.4.2",, then you can remove it using the following command: npm uninstall grunt-concurrent Then, you just need to make sure to remove the name of the plugin from your package.json file so that it is not loaded by the load-grunt-tasks plugin the next time you run a Grunt task. Running the npm prune command after removing the items from the package.json file will also remove the plugins. The prune command removes extraneous packages that are not listed in the parent package's dependencies list. See also More information on the npm version's syntax can be found at https://www. npmjs.org/doc/misc/semver.html  Also, see http://caniuse.com/ for more information on the Can I Use database Utilizing the Gruntfile.js file The Gruntfile.js file is the main configuration file for Grunt that handles all the tasks and task configurations. All the tasks and plugins are loaded using this file. In this recipe, you will create this file and will learn how to load Grunt plugins using it. Getting ready First, you need to install Node and Grunt, as described in the Installing Grunt recipe of this article. You will also have to install some Grunt plugins, as described in the Installing Grunt plugins recipe of this article. How to do it... Once you have installed Node and Grunt, follow these steps: In your Grunt project directory (the folder that contains the package.json file), create a new file, save it as Gruntfile.js, and add the following lines to it: module.exports = function(grunt) {   grunt.initConfig({     pkg: grunt.file.readJSON('package.json'),       //Add the Tasks configurations here.   }); // Define Tasks here }; This is the simplest form of the Gruntfile.js file that only contains two information variables. The next step is to load the plugins that you installed in the Installing Grunt plugins recipe. Add the following lines at the end of your Gruntfile.js file: grunt.loadNpmTasks('grunt-sass'); In the preceding line of code, grunt-sass is the name of the plugin you want to load. That is all it takes to load all the necessary plugins. The next step is to add the configurations for each task to the Gruntfile.js file. How it works... Any Grunt plugin can be loaded by adding a line of JavaScript to the Gruntfile.js file, as follows: grunt.loadNpmTasks('name-of-module'); This line should be added every time a new plugin is installed so that Grunt can access the plugin's functions. However, it is tedious to load every single plugin that you install. In addition, you will soon notice that, as your project grows, the number of configuration lines will increase as well. The Gruntfile.js file should be written in JavaScript or CoffeeScript. Grunt tasks rely on configuration data defined in a JSON object passed to the grunt.initConfig method. JavaScript Object Notation (JSON) is an alternative for XML and used for data exchange. JSON describes name-value pairs written as "name": "value". All the JSON data is separated by commas with JSON objects written inside curly brackets and JSON arrays inside square brackets. Each object can hold more than one name/value pair with each array holding one or more objects. You can also group tasks into one task. Your alias groups of tasks using the following line of code: grunt.registerTask('alias',['task1', 'task2']); There's more... Instead of loading all the required Grunt plugins one by one, you can load them automatically with the load-grunt-tasks plugin. You can install this by using the following command in the root of your project: npm install load-grunt-tasks --save-dev Then, add the following line at the very beginning of your Gruntfile.js file after module.exports: require('load-grunt-tasks')(grunt); Now, your Gruntfile.js file should look like this: module.exports = function(grunt) {   require('load-grunt-tasks')(grunt);   grunt.initConfig({     pkg: grunt.file.readJSON('package.json'),       //Add the Tasks configurations here.   }); // Define Tasks here }; The load-grunt-tasks plugin loads all the plugins specified in the package.json file. It simply loads the plugins that begin with the grunt- prefix or any pattern that you specify. This plugin will also read dependencies, devDependencies, and peerDependencies in your package.json file and load the Grunt tasks that match the provided patterns. A pattern to load specifically chosen plugins can be added as a second parameter. You can load, for instance, all the grunt-contrib tasks with the following code in your Gruntfile.js file: require('load-grunt-tasks')(grunt, {pattern: 'grunt-contrib-*'}); See also Read more about the load-grunt-tasks module at https://github.com/sindresorhus/load-grunt-task Adding a configuration definition for a plugin Any Grunt task needs a configuration definition. The configuration definitions are usually added to the Gruntfile.js file itself and are very easy to set up. In addition, it is very convenient to define and work with them because they are all written in the JSON format. This makes it very easy to spot the configurations in the plugin's documentation examples and add them to your Gruntfile.js file. In this recipe, you will learn how to add the configuration for a Grunt task. Getting ready For this recipe, you will first need to create a basic Gruntfile.js file and install the plugin you want to configure. If you want to install the grunt-example plugin, you can install it using the following command in the root of your project: npm install grunt-example --save-dev How to do it... Once you have created the basic Gruntfile.js file (also refer to the Utilizing the Gruntfile.js file recipe of this article), follow this step: A simple form of the task configuration is shown in the following code. Start by adding it to your Gruntfile.js file wrapped inside grunt.initConfig{}: example: {   subtask: {    files: {      "stylesheets/main.css":      "sass/main.scss"     }   } } How it works... If you look closely at the task configuration, you will notice the files field that specifies what files are going to be operated on. The files field is a very standard field that appears in almost all the Grunt plugins simply due to the fact that many tasks require some or many file manipulations. There's more... The Don't Repeat Yourself (DRY) principle can be applied to your Grunt configuration too. First, define the name and the path added to the beginning of the Gruntfile.js file as follows: app {  dev : "app/dev" } Using the templates is a key in order to avoid hard coded values and inflexible configurations. In addition, you should have noticed that the template has been used using the <%= %> delimiter to expand the value of the development directory: "<%= app.dev %>/css/main.css": "<%= app.dev %>/scss/main.scss"   The <%= %> delimiter essentially executes inline JavaScript and replaces values, as you can see in the following code:   "app/dev/css/main.css": "app/dev/scss/main.scss" So, put simply, the value defined in the app object at the top of the Gruntfile.js file is evaluated and replaced. If you decide to change the name of your development directory, for example, all you need to do is change the app's variable that is defined at the top of your Gruntfile.js file. Finally, it is also worth mentioning that the value for the template does not necessarily have to be a string and can be a JavaScript literal. See also You can read more about templates in the Templates section of Grunt's documentation at http://gruntjs.com/configuring- tasks#templates Adding the Sass compiler task The Sass tasks are the core task that you will need for your Sass development. It has several features and options, but at the heart of it is the Sass compiler that can compile your Sass files into CSS. By the end of this recipe, you will have a good understanding of this plugin, how to add it to your Gruntfile.js file, and how to take advantage of it. In this recipe, the grunt-contrib-sass plugin will be used. This plugin compiles your Sass code by using Ruby Sass. You should use the grunt-sass plugin to compile Sass into CSS with node-sass (LibSass). Getting ready The only requirement for this recipe is to have the grunt-contrib-sass plugin installed and loaded in your Gruntfile.js file. If you have not installed this plugin in the Installing Grunt Plugins recipe of this article, you can do this using the following command in the root of your project: npm install grunt-contrib-sass --save-dev You should also install grunt local by running the following command: npm install grunt --save-dev Finally, your project should have the file and directory, as describe in the Installing Grunt plugins recipe of this article. How to do it... An example of the Sass task configuration is shown in the following code. Start by adding it to your Gruntfile.js file wrapped inside the grunt.initConfig({}) code. Now, your Gruntfile.js file should look as follows: module.exports = function(grunt) {   grunt.initConfig({     //Add the Tasks configurations here.     sass: {                                            dist: {                                            options: {                                       style: 'expanded'         },         files: {                                         'stylesheets/main.css': 'sass/main.scss'  'source'         }       }     }   });     grunt.loadNpmTasks('grunt-contrib-sass');     // Define Tasks here    grunt.registerTask('default', ['sass']);  } Then, run the following command in your console: grunt sass The preceding command will create a new stylesheets/main.css file. Also, notice that the stylesheets/main.css.map file has also been automatically created. The Sass compiler task creates CSS sourcemaps to debug your code by default. How it works... In addition to setting up the task configuration, you should run the Grunt command to test the Sass task. When you run the grunt sass command, Grunt will look for a configuration called Sass in the Gruntfile.js file. Once it finds it, it will run the task with some default options if they are not explicitly defined. Successful tasks will end with the following message: Done, without errors. There's more... There are several other options that you can include in the Sass task. An option can also be set at the global Sass task level, so the option will be applied in all the subtasks of Sass. In addition to options, Grunt also provides targets for every task to allow you to set different configurations for the same task. In other words, if, for example, you need to have two different versions of the Sass task with different source and destination folders, you could easily use two different targets. Adding and executing targets are very easy. Adding more builds just follows the JSON notation, as shown here:    sass: {                                      // Task       dev: {                                    // Target         options: {                               // Target options           style: 'expanded'         },         files: {                                 // Dictionary of files         'stylesheets/main.css': 'sass/main.scss' // 'destination': 'source'         }       },       dist: {                               options: {                        style: 'expanded',           sourcemap: 'none'                  },         files: {                                      'stylesheets/main.min.css': 'sass/main.scss'         }       }     } In the preceding example, two builds are defined. The first one is named dev and the second is called dist. Each of these targets belongs to the Sass task, but they use different options and different folders for the source and the compiled Sass code. Moreover, you can run a particular target using grunt sass:nameOfTarget, where nameOfTarge is the name of the target that you are trying to use. So, for example, if you need to run the dist target, you will have to run the grunt sass:dist command in your console. However, if you need to run both the targets, you could simply run grunt sass and it would run both the targets sequentially. As already mentioned, the grunt-contrib-sass plugin compiles your Sass code by using Ruby Sass, and you should use the grunt-sass plugin to compile Sass to CSS with node-sass (LibSass). To switch to the grunt-sass plugin, you will have to install it locally first by running the following command in your console: npm install grunt-sass Then, replace grunt.loadNpmTasks('grunt-contrib-sass'); with grunt.loadNpmTasks('grunt-sass'); in the Gruntfile.js file; the basic options for grunt-contrib-sass and grunt-sass are very similar, so you have to change the options for the Sass task when switching to grunt-sass. Finally, notice that grunt-contrib-sass also has an option to turn Compass on. See also Please refer to Grunt's documentation for a full list of options, which is available at https://gruntjs/grunt-contrib-sass#options Also, read Grunt's documentation for more details about configuring your tasks and targets at http://gruntjs.com/configuring-tasks#task-configuration-and-targets github.com/ Summary In this article you studied about installing Grunt, installing Grunt plugins, utilizing the Gruntfile.js file, adding a configuration definition for a plugin and adding the Sass compiler task. Resources for Article: Further resources on this subject: Meeting SAP Lumira [article] Security in Microsoft Azure [article] Basic Concepts of Machine Learning and Logistic Regression Example in Mahout [article]

In this article by Sebastián Di Giuseppe, author of the book, Building a 3D game with LibGDX, describes about how to work in 3 dimensions! For which we require new camera techniques. The third dimension adds a new axis, instead of having just the x and y grid, a slightly different workflow, and lastly new render methods are required to draw our game. We'll learn the very basics of this workflow in this article for you to have a sense of what's coming, like moving, scaling, materials, environment, and some others and we are going to move systematically between them one step at a time. (For more resources related to this topic, see here.) The following topics will be covered in this article: Camera techniques Workflow LibGDX's 3D rendering API Math Camera techniques The goal of this article is to successfully learn about working with 3D as stated. In order to achieve this we will start at the basics, making a simple first person camera. We will facilitate the functions and math that LibGDX contains. Since you probably have used LibGDX more than once, you should be familiar with the concepts of the camera in 2D. The way 3D works is more or less the same, except there is a z axis now for the depth . However instead of an OrthographicCamera class, a PerspectiveCamera class is used to set up the 3D environment. Creating a 3D camera is just as easy as creating a 2D camera. The constructor of a PerspectiveCamera class requires three arguments, the field of vision, camera width and camera height. The camera width and height are known from 2D cameras, the field of vision is new. Initialization of a PerspectiveCamera class looks like this: float FoV = 67; PerspectiveCamera camera = new PerspectiveCamera(FoV, Gdx.graphics.getWidth(), Gdx.graphics.getHeight()); The first argument, field of vision, describes the angle the first person camera can see. The image above gives a good idea what the field of view is. For first person shooters values up to 100 are used. Higher than 100 confuses the player, and with a lower field of vision the player is bound to see less. Displaying a texture. We will start by doing something exciting, drawing a cube on the screen! Drawing a cube First things first! Let's create a camera. Earlier, we showed the difference between the 2D camera and the 3D camera, so let's put this to use. Start by creating a new class on your main package (ours is com.deeep.spaceglad) and name it as you like. The following imports are used on our test: import com.badlogic.gdx.ApplicationAdapter; import com.badlogic.gdx.Gdx; import com.badlogic.gdx.graphics.Color; import com.badlogic.gdx.graphics.GL20; import com.badlogic.gdx.graphics.PerspectiveCamera; import com.badlogic.gdx.graphics.VertexAttributes; import com.badlogic.gdx.graphics.g3d.*; import com.badlogic.gdx.graphics.g3d.attributes.ColorAttribute; import com.badlogic.gdx.graphics.g3d.environment.DirectionalLight; import com.badlogic.gdx.graphics.g3d.utils.ModelBuilder; Create a class member called cam of type PerspectiveCamera; public PerspectiveCamera cam; Now this camera needs to be initialized and needs to be configured. This will be done in the create method as shown below. public void create() { cam = new PerspectiveCamera(67, Gdx.graphics.getWidth(), Gdx.graphics.getHeight()); cam.position.set(10f, 10f, 10f); cam.lookAt(0,0,0); cam.near = 1f; cam.far = 300f; cam.update(); } In the above code snippet we are setting the position of the camera, and looking towards a point set at 0, 0, 0 . Next up, is getting a cube ready to draw. In 2D it was possible to draw textures, but textures are flat. In 3D, models are used. Later on we will import those models. But we will start with generated models. LibGDX offers a convenient class to build simple models such as: spheres, cubes, cylinders, and many more to choose from. Let's add two more class members, a Model and a ModelInstance. The Model class contains all the information on what to draw, and the resources that go along with it. The ModelInstance class has information on the whereabouts of the model such as the location rotation and scale of the model. public Model model; public ModelInstance instance; Add those class members. We use the overridden create function to initialize our new class members. public void create() { … ModelBuilder modelBuilder = new ModelBuilder();Material mat = new Material(ColorAttribute.createDiffuse(Color.BLUE));model = modelBuilder.createBox(5, 5, 5, mat, VertexAttributes.Usage.Position | VertexAttributes.Usage.Normal);instance = new ModelInstance(model); } We use a ModelBuilder class to create a box. The box will need a material, a color. A material is an object that holds different attributes. You could add as many as you would like. The attributes passed on to the material changes the way models are perceived and shown on the screen. We could, for example, add FloatAttribute.createShininess(8f) after the ColorAttribute class, that will make the box to shine with lights around. There are more complex configurations possible but we will leave that out of the scope for now. With the ModelBuilder class, we create a box of (5, 5, 5). Then we pass the material in the constructor, and the fifth argument are attributes for the specific box we are creating. We use a bitwise operator to combine a position attribute and a normal attribute. We tell the model that it has a position, because every cube needs a position, and the normal is to make sure the lighting works and the cube is drawn as we want it to be drawn. These attributes are passed down to openGL on which LibGDX is build. Now we are almost ready for drawing our first cube. Two things are missing, first of all: A batch to draw to. When designing 2D games in LibGDX a SpriteBatch class is used. However since we are not using sprites anymore, but rather models, we will use a ModelBatch class. Which is the equivalent for models. And lastly, we will have to create an environment and add lights to it. For that we will need two more class members: public ModelBatchmodelBatch; public Environment environment; And they are to be initialized, just like the other class members: public void create() { .... modelBatch = new ModelBatch(); environment = new Environment(); environment.set(new ColorAttribute(ColorAttribute.AmbientLight, 0.4f, 0.4f, 0.4f, 1f)); environment.add(new DirectionalLight().set(0.8f, 0.8f, 0.8f, - 1f, -0.8f, -0.2f)); } Here we add two lights, an ambient light, which lights up everything that is being drawn (a general light source for all the environment), and a directional light, which has a direction (most similar to a "sun" type of source). In general, for lights, you can experiment directions, colors, and different types. Another type of light would be PointLight and it can be compared to a flashlight. Both lights start with 3 arguments, for the color, which won't make a difference yet as we don't have any textures. The directional lights constructor is followed by a direction. This direction can be seen as a vector. Now we are all set to draw our environment and the model in it @Override public void render() { Gdx.gl.glViewport(0, 0, Gdx.graphics.getWidth(), Gdx.graphics.getHeight()); Gdx.gl.glClear(GL20.GL_COLOR_BUFFER_BIT | GL20.GL_DEPTH_BUFFER_BIT); modelBatch.begin(cam); modelBatch.render(instance, environment); modelBatch.end(); } It directly renders our cube. The ModelBatch catch behaves just like a SpriteBatch, as can be seen if we run it, it has to be started (begin), then ask for it to render and give them the parameters (models and environment in our case), and then make it stop. We should not forget to release any resources that our game allocated. The model we created allocates memory that should be disposed of. @Override public void dispose() { model.dispose(); } Now we can look at our beautiful cube! It's only very static and empty. We will add some movement to it in our next subsection! Translation Translating rotating and scaling are a bit different to that of a 2D game. It's slightly more mathematical. The easier part are vectors, instead of a vector2D, we can now use a vector3D, which is essentially the same, just that, it adds another dimension. Let's look at some basic operations of 3D models. We will use the cube that we previously created. With translation we are able to move the model along all three the axis. Let's create a function that moves our cube along the x axis. We add a member variable to our class to store the position in for now. A Vector3 class. Vector3 position = new Vector3(); private void movement() { instance.transform.getTranslation(position); position.x += Gdx.graphics.getDeltaTime(); instance.transform.setTranslation(position); } The above code snippet retrieves the translation, adds the delta time to the x attribute of the translation. Then we set the translation of the ModelInstance. The 3D library returns the translation a little bit different than normally. We pass a vector, and that vector gets adjusted to the current state of the object. We have to call this function every time the game updates. So therefore we put it in our render loop before we start drawing. @Override public void render() { movement(); ... } It might seem like the cube is moving diagonally, but that's because of the angle of our camera. In fact it's' moving towards one face of the cube. That was easy! It's only slightly annoying that it moves out of bounds after a short while. Therefor we will change the movement function to contain some user input handling. private void movement() { instance.transform.getTranslation(position); if(Gdx.input.isKeyPressed(Input.Keys.W)){ position.x+=Gdx.graphics.getDeltaTime(); } if(Gdx.input.isKeyPressed(Input.Keys.D)){ position.z+=Gdx.graphics.getDeltaTime(); } if(Gdx.input.isKeyPressed(Input.Keys.A)){ position.z-=Gdx.graphics.getDeltaTime(); } if(Gdx.input.isKeyPressed(Input.Keys.S)){ position.x-=Gdx.graphics.getDeltaTime(); } instance.transform.setTranslation(position); } The rewritten movement function retrieves our position, updates it based on the keys that are pressed, and sets the translation of our model instance. Rotation Rotation is slightly different from 2D. Since there are multiple axes on which we can rotate, namely the x, y, and z axis. We will now create a function to showcase the rotation of the model. First off let us create a function in which  we can rotate an object on all axis private void rotate() { if (Gdx.input.isKeyPressed(Input.Keys.NUM_1)) instance.transform.rotate(Vector3.X, Gdx.graphics.getDeltaTime() * 100); if (Gdx.input.isKeyPressed(Input.Keys.NUM_2)) instance.transform.rotate(Vector3.Y, Gdx.graphics.getDeltaTime() * 100); if (Gdx.input.isKeyPressed(Input.Keys.NUM_3)) instance.transform.rotate(Vector3.Z, Gdx.graphics.getDeltaTime() * 100); } And let's not forget to call this function from the render loop, after we call the movement function @Override public void render() { ... rotate(); } If we press the number keys 1, 2 or 3, we can rotate our model. The first argument of the rotate function is the axis to rotate on. The second argument is the amount to rotate. These functions are to add a rotation. We can also set the value of an axis, instead of add a rotation, with the following function: instance.transform.setToRotation(Vector3.Z, Gdx.graphics.getDeltaTime() * 100); However say, we want to set all three axis rotations at the same time, we can't simply call setToRotation function three times in a row for each axis, as they eliminate any other rotation done before that. Luckily LibGDX has us covered with a function that is able to take all three axis. float rotation; private void rotate() { rotation = (rotation + Gdx.graphics.getDeltaTime() * 100) % 360; instance.transform.setFromEulerAngles(0, 0, rotation); } The above function will continuously rotate our cube. We face one last problem. We can't seem to move the cube! The setFromEulerAngles function clears all the translation and rotation properties. Lucky for us the setFromEulerAngles returns a Matrix4 type, so we can chain and call another function from it. A function which translates the matrix for example. For that we use the trn(x,y,z) function. Short for translate. Now we can update our rotation function, although it also translates. instance.transform.setFromEulerAngles(0, 0, rotation).trn(position.x, position.y, position.z); Now we can set our cube to a rotation, and translate it! These are the most basic operations which we will use a lot throughout the book. As you can see this function does both the rotation and the translation. So we can remove the last line in our movement function instance.transform.setTranslation(position); Our latest rotate function looks like the following: private void rotate() { rotation = (rotation + Gdx.graphics.getDeltaTime() * 100) % 360; instance.transform.setFromEulerAngles(0, 0, rotation).trn(position.x, position.y, position.z); } The setFromEulerAngles function will be extracted to a function of its own, as it serves multiple purposes now and is not solely bound to our rotate function. private void updateTransformation(){ instance.transform.setFromEulerAngles(0, 0, rotation).trn(position.x, position.y, position.z).scale(scale,scale,scale); } This function should be called after we've calculated our rotation and translation public void render() { rotate(); movement(); updateTransformation(); ... } Scaling We've almost had all of the transformations we can apply to models. The last one being described in this book is the scaling of a model. LibGDX luckily contains all the required functions and methods for this. Let's extend our previous example and make our box growing and shrinking over time. We first create a function that increments and subtracts from a scale variable. boolean increment;float scale = 1; void scale(){ if(increment) { scale = (scale + Gdx.graphics.getDeltaTime()/5); if (scale >= 1.5f) { increment = false; } else { scale = (scale - Gdx.graphics.getDeltaTime()/5); if(scale <= 0.5f) increment = true; } } Now to apply this scaling we can adjust our updateTransformation function to include the scaling. private void updateTransformation(){ instance.transform.setFromEulerAngles(0, 0, rotation).trn(position.x, position.y, position.z).scale(scale,scale,scale); } Our render method should now include the scaling function as well public void render() { rotate(); movement(); scale(); updateTransformation(); ... } And there you go, we can now successfully move, rotate and scale our cube! Summary In this article we learned about the workflow of LibGDX 3D API. We are now able to apply multiple kinds of transformations to a model, and understand the differences between 2D and 3D. We also learned how to apply materials to models, which will change the appearance of the model and lets us create cool effects. Note that there's plenty more information that you can learn about 3D and a lot of practice to go with it to fully understand it. There's also subjects not covered here, like how to create your own materials, and how to make and use of shaders. There's plenty room for learning and experimenting. In the next article we will start on applying the theory that's learned in this article, and start working towards an actual game! We will also go more in depth on the environment and lights, as well as collision detection. So plenty to look forward to. Resources for Article: Further resources on this subject: 3D Websites [Article] Your 3D World [Article] Using 3D Objects [Article]

After working for over 1.5 years on the Differentiable Programming Mega-Proposal, Richard Wei, a developer at Google Brain, and his team submitted the proposal on the Swift Evolution forum on Thursday last week. This proposal aims to “push Swift's capabilities to the next level in numerics and machine learning” by introducing differentiable programming as a new language feature in Swift. It is a part of the Swift for TensorFlow project under which the team is integrating TensorFlow directly into the language to offer developers a next-generation platform for machine learning. What is differentiable programming With the increasing sophistication in deep learning models and the introduction of modern deep learning frameworks, many researchers have started to realize that building neural networks is very similar to programming. Yann LeCun, VP and Chief AI Scientist at Facebook, calls differentiable programming “a little more than a rebranding of the modern collection Deep Learning techniques, the same way Deep Learning was a rebranding of the modern incarnations of neural nets with more than two layers.” He compares it with regular programming, with the only difference that the resulting programs are “parameterized, automatically differentiated, and trainable/optimizable.” Many also say that differentiable programming is a different name for automatic differentiation, a collection of techniques to numerically evaluate the derivative of a function. It can be seen as a new programming paradigm in which programs can be differentiated throughout. Check out the paper “Demystifying Differentiable Programming: Shift/Reset the Penultimate Backpropagation” to get a better understanding of differentiable programming. Why differential programming is proposed in Swift Swift is an expressive, high-performance language, which makes it a perfect candidate for numerical applications. According to the proposal authors, first-class support for differentiable programming in Swift will allow safe and powerful machine learning development. The authors also believe that this is a “big step towards high-level numerical computing support.” With this proposal, they aim to make Swift a “real contender in the numerical computing and machine learning landscape.” Here are some of the advantages of adding first-class support for differentiable programming in Swift: Better language coverage: First-class differentiable programming support will enable differentiation to work smoothly with other Swift features. This will allow developers to code normally without being restricted to a subset of Swift. Enable extensibility: This will provide developers an “extensible differentiable programming system.” They will be able to create custom differentiation APIs by leveraging primitive operators defined in the standard library and supported by the type system. Static warnings and errors: This will enable the compiler to statically identify the functions that cannot be differentiated or will give a zero derivative. It will then be able to give a non-differentiability error or warning. This will improve productivity by making common runtime errors in machine learning directly debuggable without library boundaries. Some of the components that will be added in Swift under this proposal are: The Differentiable protocol: This is a standard library protocol that will generalize all data structures that can be a parameter or result of a differentiable function. The @differentiable declaration attribute: This will be used to mark all the function-like declarations as differentiable. The @differentiable function types: This is a subtype of normal function types with a different runtime representation and calling convention. Differentiable function types will have differentiable parameters and results. Differential operators: These are the core differentiation APIs that take ‘@differentiable’ functions as inputs and return derivative functions or compute derivative values. @differentiating and @transposing attributes: These attributes are for declaring custom derivative function for some other function declaration. This proposal sparked a discussion on Hacker News. Many developers were excited about bringing differentiable programming support in the Swift core. A user commented, “This is actually huge. I saw a proof of concept of something like this in Haskell a few years back, but it's amazing it see it (probably) making it into the core of a mainstream language. This may let them capture a large chunk of the ML market from Python - and hopefully, greatly improve ML APIs while they're at it.” Some felt that a library could have served the purpose. “I don't see why a well-written library could not serve the same purpose. It seems like a lot of cruft. I doubt, for example, Python would ever consider adding this and it's the de facto language that would benefit the most from something like this - due to the existing tools and communities. It just seems so narrow and not at the same level of abstraction that languages typically sit at. I could see the language supporting higher-level functionality so a library could do this without a bunch of extra work (such as by some reflection),” a user added. Users also discussed another effort that goes along the lines of this project: Julia Zygote, which is a working prototype for source-to-source automatic differentiation. A user commented, “Yup, work is continuing apace with Julia’s next-gen Zygote project. Also, from the GP’s thought about applications beyond DL, my favorite examples so far are for model-based RL and Neural ODEs.” To know more in detail, check out the proposal: Differentiable Programming Mega-Proposal. Other news in programming Why Perl 6 is considering a name change? The Julia team shares its finalized release process with the community TypeScript 3.6 releases with stricter generators, new functions in TypeScript playground, better Unicode support for identifiers, and more

In this article by Anshul Verma and Jitendra Zaa, author of the book Apex Design Patterns, we will discuss some problems that can occur mainly during the creation of class instances and how we can write the code for the creation of objects in a more simple, easy to maintain, and scalable way. (For more resources related to this topic, see here.) In this article, we will discuss the the factory method creational design pattern. Often, we find that some classes have common features (behavior) and can be considered classes of the same family. For example, multiple payment classes represent a family of payment services. Credit card, debit card, and net banking are some of the examples of payment classes that have common methods, such as makePayment, authorizePayment, and so on. Using the factory method pattern, we can develop controller classes, which can use these payment services, without knowing the actual payment type at design time. The factory method pattern is a creational design pattern used to create objects of classes from the same family without knowing the exact class name at design time. Using the factory method pattern, classes can be instantiated from the common factory method. The advantage of using this pattern is that it delegates the creation of an object to another class and provides a good level of abstraction. Let's learn this pattern using the following example: The Universal Call Center company is new in business and provides free admin support to customers to resolve issues related to their products. A call center agent can provide some information about the product support; for example, to get the Service Level Agreement (SLA) or information about the total number of tickets allowed to open per month. A developer came up with the following class: public class AdminBasicSupport{ /** * return SLA in hours */ public Integer getSLA() { return 40; } /** * Total allowed support tickets allowed every month */ public Integer allowedTickets() { // As this is basic support return 9999; } } Now, to get the SLA of AdminBasicSupport, we need to use the following code every time: AdminBasicSupport support = new AdminBasicSupport(); System.debug('Support SLA is - '+support.getSLA()); Output - Support SLA is – 40 The "Universal Call Centre" company was doing very well, and in order to grow the business and increase the profit, they started the premium support for customers who were willing to pay for cases and get a quick support. To make them special from the basic support, they changed the SLA to 12 hours and maximum 50 cases could be opened in one month. A developer had many choices to make this happen in the existing code. However, instead of changing the existing code, they created a new class that would handle only the premium support-related functionalities. This was a good decision because of the single responsibility principle. public class AdminPremiumSupport{ /** * return SLA in hours */ public Integer getSLA() { return 12; } /** * Total allowed support tickets allowed every month is 50 */ public Integer allowedTickets() { return 50; } } Now, every time any information regarding the SLA or allowed tickets per month is needed, the following Apex code can be used: if(Account.supportType__c == 'AdminBasic') { AdminBasicSupport support = new AdminBasicSupport(); System.debug('Support SLA is - '+support.getSLA()); }else{ AdminPremiumSupport support = new AdminPremiumSupport(); System.debug('Support SLA is - '+support.getSLA()); } As we can see in the preceding example, instead of adding some conditions to the existing class, the developer decided to go with a new class. Each class has its own responsibility, and they need to be changed for only one reason. If any change is needed in the basic support, then only one class needs to be changed. As we all know that this design principle is known as the Single Responsibility Principle. Business was doing exceptionally well in the call center, and they planned to start the golden and platinum support as well. Developers started facing issues with the current approach. Currently, they have two classes for the basic and premium support and requests for two more classes were in the pipeline. There was no guarantee that the support type will not remain the same in future. Because of every new support type, a new class is needed; and therefore, the previous code needs to be updated to instantiate these classes. The following code will be needed to instantiate these classes: if(Account.supportType__c == 'AdminBasic') { AdminBasicSupport support = new AdminBasicSupport(); System.debug('Support SLA is - '+support.getSLA()); }else if(Account.supportType__c == 'AdminPremier') { AdminPremiumSupport support = new AdminPremiumSupport(); System.debug('Support SLA is - '+support.getSLA()); }else if(Account.supportType__c == 'AdminGold') { AdminGoldSupport support = new AdminGoldSupport(); System.debug('Support SLA is - '+support.getSLA()); }else{ AdminPlatinumSupport support = new AdminPlatinumSupport(); System.debug('Support SLA is - '+support.getSLA()); } We are only considering the getSLA() method, but in a real application, there can be other methods and scenarios as well. The preceding code snippet clearly depicts the code duplicity and maintenance nightmare. The following image shows the overall complexity of the example that we are discussing: Although they are using a separate class for each support type, an introduction to a new support class will lead to changes in the code in all existing code locations where these classes are being used. The development team started brainstorming to make sure that the code is capable to extend easily in future with the least impact on the existing code. One of the developers came up with a suggestion to use an interface for all support classes so that every class can have the same methods and they can be referred to using an interface. The following interface was finalized to reduce the code duplicity: public Interface IAdminSupport{ Integer getSLA() ; Integer allowedTickets(); } Methods defined within an interface have no access modifiers and just contain their signatures. Once an interface was created, it was time to update existing classes. In our case, only one line needed to be changed and the remaining part of the code was the same because both the classes already have the getSLA() and allowedTickets() methods. Let's take a look at the following line of code: public class AdminPremiumSupport{ This will be changed to the following code: public class AdminBasicSupportImpl implements IAdminSupport{ The following line of code is as follows: public class AdminPremiumSupport{ This will be changed to the following code: public class AdminPremiumSupportImpl implements IAdminSupport{ In the same way, the AdminGoldSupportImpl and AdminPlatinumSupportImpl classes are written. A class diagram is a type of Unified Modeling Language (UML), which describes classes, methods, attributes, and their relationships, among other objects in a system. You can read more about class diagrams at https://en.wikipedia.org/wiki/Class_diagram. The following image shows a class diagram of the code written by developers using an interface: Now, the code to instantiate different classes of the support type can be rewritten as follows: IAdminSupport support = null; if(Account.supportType__c == 'AdminBasic') { support = new AdminBasicSupportImpl(); }else if(Account.supportType__c == 'AdminPremier') { support = new AdminPremiumSupportImpl(); }else if(Account.supportType__c == 'AdminGold') { support = new AdminGoldSupportImpl(); }else{ support = new AdminPlatinumSupportImpl(); } System.debug('Support SLA is - '+support.getSLA()); There is no switch case statement in Apex, and that's why multiple if and else statements are written. As per the product team, a new compiler may be released in 2016 and it will be supported. You can vote for this idea at https://success.salesforce.com/ideaView?id=08730000000BrSIAA0. As we can see, the preceding code is minimized to create a required instance of a concrete class, and then uses an interface to access methods. This concept is known as program to interface. This is one of the most recommended OOP principles suggested to be followed. As interfaces are kinds of contracts, we already know which methods will be implemented by concrete classes, and we can completely rely on the interface to call them, which hides their complex implementation and logic. It has a lot of advantages and a few of them are loose coupling and dependency injection. A concrete class is a complete class that can be used to instantiate objects. Any class that is not abstract or an interface can be considered a concrete class. We still have one problem in the previous approach. The code to instantiate concrete classes is still present at many locations and will still require changes if a new support type is added. If we can delegate the creation of concrete classes to some other class, then our code will be completely independent of the existing code and new support types. This concept of delegating decisions and creation of similar types of classes is known as the factory method pattern. The following class can be used to create concrete classes and will act as a factory: /** * This factory class is used to instantiate concrete class * of respective support type * */ public class AdminSupportFactory { public static IAdminSupport getInstance(String supporttype){ IAdminSupport support = null; if(supporttype == 'AdminBasic') { support = new AdminBasicSupportImpl(); }else if(supporttype == 'AdminPremier') { support = new AdminPremiumSupportImpl(); }else if(supporttype == 'AdminGold') { support = new AdminGoldSupportImpl(); }else if(supporttype == 'AdminPlatinum') { support = new AdminPlatinumSupportImpl(); } return support ; } } In the preceding code, we only need to call the getInstance(string) method, and this method will take a decision and return the actual implementation. As a return type is an interface, we already know the methods that are defined, and we can use the method without actually knowing its implementation. This is a very good example of abstraction. The final class diagram of the factory method pattern that we discussed will look like this: The following code snippet can be used repeatedly by any client code to instantiate a class of any support type: IAdminSupport support = AdminSupportFactory.getInstance ('AdminBasic'); System.debug('Support SLA is - '+support.getSLA()); Output : Support SLA is – 40 Reflection in Apex The problem with the preceding design is that whenever a new support needs to be added, we need to add a condition to AdminSupportFactory. We can store the mapping between a support type and its concrete class name in Custom setting. This way, whenever a new concrete class is added, we don't even need to change the factory class and a new entry needs to be added to custom setting. Consider custom setting created by the Support_Type__c name with the Class_Name__c field name of the text type with the following records: Name Class name AdminBasic AdminBasicSupportImpl AdminGolden AdminGoldSupportImpl AdminPlatinum AdminPlatinumSupportImpl AdminPremier AdminPremiumSupportImpl However, using reflection, the AdminSupportFactory class can also be rewritten to instantiate service types at runtime as follows: /** * This factory class is used to instantiate concrete class * of respective support type * */ public class AdminSupportFactory { public static IAdminSupport getInstance(String supporttype) { //Read Custom setting to get actual class name on basis of Support type Support_Type__c supportTypeInfo = Support_Type__c.getValues(supporttype); //from custom setting get appropriate class name Type t = Type.forName(supportTypeInfo.Class_Name__c); IAdminSupport retVal = (IAdminSupport)t.newInstance(); return retVal; } } In the preceding code, we are using the Type system class. This is a very powerful class used to instantiate a new class at runtime. It has the following two important methods: forName: This returns a type that is equivalent to a string passed newInstance: This creates a new object for a specified type Inspecting classes, methods, and variables at runtime without knowing a class name, or instantiating a new object and invoking methods at runtime is known as Reflection in computer science. One more advantage of using the factory method, custom setting, and reflection together is that if in future one of the support types need to be replaced by another service type permanently, then we need to simply change the appropriate mapping in custom setting without any changes in the code. Summary In this article, we discussed how to deal with various situations while instantiating objects using design patterns, using the factory method. Resources for Article: Further resources on this subject: Getting Your APEX Components Logic Right[article] AJAX Implementation in APEX[article] Custom Coding with Apex[article]