How-To Tutorials - Programming

A design pattern is a reusable solution to a recurring problem. The term is really broad in its definition and can span multiple domains of an application. However, the term is often associated with a well-known set of object-oriented patterns that were popularized in the 90s by the book, Design Patterns: Elements of Reusable Object- Oriented Software, Pearson Education, by the almost legendary Gang of Four (GoF): Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides. This article is an excerpt from the book Node.js Design Patterns, Third Edition by Mario Casciaro and Luciano Mammino – a comprehensive guide for learning proven patterns, techniques, and tricks to take full advantage of the Node.js platform. In this article, we’ll look at the behavior of components in software design. We’ll learn how to combine objects and how to define the way they communicate so that the behavior of the resulting structure becomes extensible, modular, reusable, and adaptable. After introducing all the behavioral design patterns, we will dive deep into the details of the strategy pattern. Now, it's time to roll up your sleeves and get your hands dirty with some behavioral design patterns. Types of Behavioral Design Patterns The Strategy pattern allows us to extract the common parts of a family of closely related components into a component called the context and allows us to define strategy objects that the context can use to implement specific behaviors. The State pattern is a variation of the Strategy pattern where the strategies are used to model the behavior of a component when under different states. The Template pattern, instead, can be considered the "static" version of the Strategy pattern, where the different specific behaviors are implemented as subclasses of the template class, which models the common parts of the algorithm. The Iterator pattern provides us with a common interface to iterate over a collection. It has now become a core pattern in Node.js. JavaScript offers native support for the pattern (with the iterator and iterable protocols). Iterators can be used as an alternative to complex async iteration patterns and even to Node.js streams. The Middleware pattern allows us to define a modular chain of processing steps. This is a very distinctive pattern born from within the Node.js ecosystem. It can be used to preprocess and postprocess data and requests. The Command pattern materializes the information required to execute a routine, allowing such information to be easily transferred, stored, and processed. The Strategy Pattern The Strategy pattern enables an object, called the context, to support variations in its logic by extracting the variable parts into separate, interchangeable objects called strategies. The context implements the common logic of a family of algorithms, while a strategy implements the mutable parts, allowing the context to adapt its behavior depending on different factors, such as an input value, a system configuration, or user preferences. Strategies are usually part of a family of solutions and all of them implement the same interface expected by the context. The following figure shows the situation we just described: Figure 1: General structure of the Strategy pattern Figure 1 shows you how the context object can plug different strategies into its structure as if they were replaceable parts of a piece of machinery. Imagine a car; its tires can be considered its strategy for adapting to different road conditions. We can fit winter tires to go on snowy roads thanks to their studs, while we can decide to fit high-performance tires for traveling mainly on motorways for a long trip. On the one hand, we don't want to change the entire car for this to be possible, and on the other, we don't want a car with eight wheels so that it can go on every possible road. The Strategy pattern is particularly useful in all those situations where supporting variations in the behavior of a component requires complex conditional logic (lots of if...else or switch statements) or mixing different components of the same family. Imagine an object called Order that represents an online order on an e-commerce website. The object has a method called pay() that, as it says, finalizes the order and transfers the funds from the user to the online store. To support different payment systems, we have a couple of options: Use an ..elsestatement in the pay() method to complete the operation based on the chosen payment option Delegate the logic of the payment to a strategy object that implements the logic for the specific payment gateway selected by the user In the first solution, our Order object cannot support other payment methods unless its code is modified. Also, this can become quite complex when the number of payment options grows. Instead, using the Strategy pattern enables the Order object to support a virtually unlimited number of payment methods and keeps its scope limited to only managing the details of the user, the purchased items, and the relative price while delegating the job of completing the payment to another object. Let's now demonstrate this pattern with a simple, realistic example. Multi-format configuration objects Let's consider an object called Config that holds a set of configuration parameters used by an application, such as the database URL, the listening port of the server, and so on. The Config object should be able to provide a simple interface to access these parameters, but also a way to import and export the configuration using persistent storage, such as a file. We want to be able to support different formats to store the configuration, for example, JSON, INI, or YAML. By applying what we learned about the Strategy pattern, we can immediately identify the variable part of the Config object, which is the functionality that allows us to serialize and deserialize the configuration. This is going to be our strategy. Creating a new module Let's create a new module called config.js, and let's define the generic part of our configuration manager: import { promises as fs } from 'fs' import objectPath from 'object-path' export class Config { constructor (formatStrategy) {                           // (1) this.data = {} this.formatStrategy = formatStrategy } get (configPath) {                                       // (2) return objectPath.get(this.data, configPath) } set (configPath, value) {                                // (2) return objectPath.set(this.data, configPath, value) } async load (filePath) {                                  // (3) console.log(`Deserializing from ${filePath}`) this.data = this.formatStrategy.deserialize( await fs.readFile(filePath, 'utf-8') ) } async save (filePath) {                                  // (3) console.log(`Serializing to ${filePath}`) await fs.writeFile(filePath, this.formatStrategy.serialize(this.data)) } } This is what's happening in the preceding code: In the constructor, we create an instance variable called data to hold the configuration data. Then we also store formatStrategy, which represents the component that we will use to parse and serialize the data. We provide two methods, set()and get(), to access the configuration properties using a dotted path notation (for example, property.subProperty) by leveraging a library called object-path (nodejsdp.link/object-path). The load() and save() methods are where we delegate, respectively, the deserialization and serialization of the data to our strategy. This is where the logic of the Config class is altered based on the formatStrategy passed as an input in the constructor. As we can see, this very simple and neat design allows the Config object to seamlessly support different file formats when loading and saving its data. The best part is that the logic to support those various formats is not hardcoded anywhere, so the Config class can adapt without any modification to virtually any file format, given the right strategy. Creating format Strategies To demonstrate this characteristic, let's now create a couple of format strategies in a file called strategies.js. Let's start with a strategy for parsing and serializing data using the INI file format, which is a widely used configuration format (more info about it here: nodejsdp.link/ini-format). For the task, we will use an npm package called ini (nodejsdp.link/ini): import ini from 'ini' export const iniStrategy = { deserialize: data => ini.parse(data), serialize: data => ini.stringify(data) } Nothing really complicated! Our strategy simply implements the agreed interface, so that it can be used by the Config object. Similarly, the next strategy that we are going to create allows us to support the JSON file format, widely used in JavaScript and in the web development ecosystem in general: export const jsonStrategy = { deserialize: data => JSON.parse(data), serialize: data => JSON.stringify(data, null, '  ') } Now, to show you how everything comes together, let's create a file named index.js, and let's try to load and save a sample configuration using different formats: import { Config } from './config.js' import { jsonStrategy, iniStrategy } from './strategies.js' async function main () { const iniConfig = new Config(iniStrategy) await iniConfig.load('samples/conf.ini') iniConfig.set('book.nodejs', 'design patterns') await iniConfig.save('samples/conf_mod.ini') const jsonConfig = new Config(jsonStrategy) await jsonConfig.load('samples/conf.json') jsonConfig.set('book.nodejs', 'design patterns') await jsonConfig.save('samples/conf_mod.json') } main() Our test module reveals the core properties of the Strategy pattern. We defined only one Config class, which implements the common parts of our configuration manager, then, by using different strategies for serializing and deserializing data, we created different Config class instances supporting different file formats. The example we've just seen shows us only one of the possible alternatives that we had for selecting a strategy. Other valid approaches might have been the following: Creating two different strategy families: One for the deserialization and the other for the serialization. This would have allowed reading from a format and saving to another. Dynamically selecting the strategy: Depending on the extension of the file provided; the Config object could have maintained a map extension → strategy and used it to select the right algorithm for the given extension. As we can see, we have several options for selecting the strategy to use, and the right one only depends on your requirements and the tradeoff in terms of features and the simplicity you want to obtain. Furthermore, the implementation of the pattern itself can vary a lot as well. For example, in its simplest form, the context and the strategy can both be simple functions: function context(strategy) {...} Even though this may seem insignificant, it should not be underestimated in a programming language such as JavaScript, where functions are first-class citizens and used as much as fully-fledged objects. Between all these variations, though, what does not change is the idea behind the pattern; as always, the implementation can slightly change but the core concepts that drive the pattern are always the same. Summary In this article, we dive deep into the details of the strategy pattern, one of the Behavioral Design Patterns in Node.js. Learn more in the book, Node.js Design Patterns, Third Edition by Mario Casciaro and Luciano Mammino. About the Authors Mario Casciaro is a software engineer and entrepreneur. Mario worked at IBM for a number of years, first in Rome, then in Dublin Software Lab. He currently splits his time between Var7 Technologies-his own software company-and his role as lead engineer at D4H Technologies where he creates software for emergency response teams. Luciano Mammino wrote his first line of code at the age of 12 on his father's old i386. Since then he has never stopped coding. He is currently working at FabFitFun as principal software engineer where he builds microservices to serve millions of users every day.

Visual Studio Code has become a very popular code editor for Angular developers, particularly those running the Angular CLI. Features such as Syntax highlighting and autocomplete, provision to debug code right in the editor, built-in Git commands and, support for extensions make VSCode among the most popular code editors around. Image Source: TRIPLEBYTE In this post, I am going to look at 5 VSCode extensions that are useful for Angular developers. #1 angular2-shortcuts If you have an Angular CLI application running on your local host, in the app folder, you have the app component that is dynamically generated by the Angular CLI. As an Angular developer, you will be working on this component and quite often switching between the html, css, and the ts file. When the application grows, you'll be switching between these three files from the individual components a lot more. For that, you have a useful extension called angular2-switcher. If you install this extension, you will get access to keyboard shortcuts to quickly navigate  the individual files. File Shortcut app.component.html Shift+Alt+u app.component.css Shift+Alt+i app.component.ts Shift+Alt+o app.component.spec.ts Shift+Alt+p The table above lists  four keyboard-shortcuts to switch between CSS, HTML, the TS file for testing and the TS file of the component itself. The letters—u, i, o and p—are very close together to make it fast to switch between the individual files. #2 Angular Language Service In Angular, if you add a name to the app component and try to render it inside of the HTML template, VSCode won’t render the name to auto-completion out of the box and needs an extension for added functionality. As an Angular developer, you want access to the inside of a template. You can use the Angular Language Service extension, which will add auto-completion. If you enable it and go back to the HTML file, you'll see if the name will populate in autocomplete list as soon as you start typing. The same would happen for the title and, for that matter, anything that is created inside of the app component; you have access to the inside of the template. If you create a simple function that returns a string, then you'll have access to it as well thanks to Angular Language Service extension. #3 json2ts The other things you will work very often in Angular are endpoints that return JSON data. For the JSON data, you will need to create a user interface. You can do it manually but if you have a massive object, then it would take you some time. Luckily, a VSCode extension can automate this for you. json2ts isn’t Angular specific and works whenever you're working with TypeScript. Json2ts comes handy when you have to create a TypeScript interface from a JSON object. #4 Bookmarks Bookmark comes handy when you're working with long files. If you want to work on a little block of code, then you need to check something at the top and then go back to the place you were before. With Bookmark, you can easily put a little marker by pressing Alt+Ctrl+K, you'll see a blue marker at the place. If you go to the top of the code where all your variables are stored, you can do the same thing—Alt+Ctrl+K. You can use Alt+Ctrl+J and Alt+Ctrl+L to jump between these two markers. When you're working on a longer file and want to quickly jump to a specific section, you can put as many of these markers as you like. Action Shortcut Set bookmark/ Remove Alt+Ctrl+K Go to previous bookmark Alt+Ctrl+J Go to next bookmark Alt+Ctrl+L There are more shortcuts to this. You can go to the menu, type in bookmarks, and you’ll see all the other keyboard shortcuts related to this extension. Setting, removing and going to the next and previous bookmark are the most useful shortcuts. #5 Guides I'm sure you came across the issue when you're looking at long codes of HTML and you're wondering, where does this tag start and end? Which div is it disclosing? Wouldn’t it be nice to have some connection between the opening and closing tags? You need some sort of rules and that's exactly what Guides does. After installing the Guides extension, vertical lines connect the opening and closing divs and help you to visualize correct indentation as shown below. Guides has many settings as well. You can change the colors or the thickness of the lines for example. These VSCode extensions improve Angular Development Workflow and I believe you will find them useful too. I know there are many more useful extensions, which you use every day. I would love to hear about them. Author Bio Aditya Modi is the CEO of TOPS Infosolutions, a Mobile and Web development company. With the right allocation of resources and emerging technology, he provides innovative solutions to businesses worldwide to solve their business and engineering problems. An avid reader, he values great books and calls them his source of motivation. You may reach out to him on LinkedIn. The January 2019 release of Visual Studio code v1.31 is out React Native Vs Ionic : Which one is the better mobile app development framework? Code completion suggestions via IntelliCode comes to C++ in Visual Studio 2019

Open source software in general, and JavaScript in particular, can seem like a place where boom and bust is the rule of law: rapid growth before everyone moves on to the next big thing. But Node.js is different. Although it certainly couldn’t be described as new, and it's growth hasn't been dramatic by any measure, over the last few years it has managed to push itself forward as one of the most widely used JavaScript tools on the planet. Do you want to learn Node.js? Popularity, however can only tell you so much. The key question, if you’re reading this, is whether you should learn Node.js. So, to help you decide if it’s time to learn the JavaScript library, here’s a list of the biggest reasons why you should start learning Node.js... Learn everything you need to know about Node.js with Packt's Node.js Complete Reference Guide Book Learning Path. Node.js lets you write JavaScript on both client and server Okay, let’s get the obvious one out of the way first: Node.js is worth learning because it allows you to write JavaScript on the server. This has arguably transformed the way we think about JavaScript. Whereas in the past it was a language specifically written on the client, backed by the likes of PHP and Java, it’s now a language that you can use across your application. Read next: The top 5 reasons Node.js could topple Java This is important because it means teams can work much more efficiently together. Using different languages for backend and frontend is typically a major source of friction. Unless you have very good polyglot developers, a team is restricted to their core skills, while tooling is also more inflexible. If you’re using JavaScript across the stack, it’s easier to use a consistent toolchain. From a personal perspective, learning Node.js is a great starting point for full stack development. In essence, it's like an add-on that immediately expands what you can do with JavaScript. In terms of your career, then, it could well make you an invaluable asset to a development team. Read next: How is Node.js changing web development? Node.js allows you to build complex and powerful applications without writing complex code Another strong argument for Node.js is that it is built for performance. This is because of 2 important things - Node.js' asynchronous-driven architecture, and the fact that it uses the V8 JavaScript engine. The significance of this is that V8 is one of the fastest implementations of JavaScript, used to power many of Google’s immensely popular in-browser products (like Gmail). Node.js is powerful because it employs an asynchronous paradigm for handling data between client and server. To clarify what this means, it’s worth comparing to the typical application server model that uses blocking I/O - in this instance, the application has to handle each request sequentially, suspending threads until they can be processed. This can add complexity to an application and, of course, slows an application down. In contrast, Node.js allows you to use non-blocking I/O in which threads (in this case sequential, not concurrent), which can manage multiple requests. If one can’t be processed, it’s effectively ‘withheld’ as a promise, which means it can be executed later without holding up other threads. This means Node.js can help you build applications of considerable complexity without adding to the complexity of your code. Node.js is well suited to building microservices Microservices have become a rapidly growing architectural style that offer increased agility and flexibility over the traditional monolith. The advantages of microservices are well documented, and whether or not they’re right for you now, it’s likely that they’re going to dominate the software landscape as the world moves away from monolithic architecture. This fact only serves to strengthen the argument that you should learn Node.js because the library is so well suited to developing in this manner. This is because it encourages you to develop in a modular and focused manner, quite literally using specific modules to develop an application. This is distinct and almost at odds with the monolithic approach to software architecture. At this point, it’s probably worth highlighting that it’s incredibly easy to package and publish the modules you build thanks to npm (node package manager). So, even if you haven’t yet worked with microservices, learning Node.js is a good way to prepare yourself for a future where they are going to become even more prevalent. Node.js can be used for more than just web development We know by now that Node.js is flexible. But it’s important to recognise that its flexibility means it can be used for a wide range of different purposes. Yes, the library's community are predominantly building applications for the web, but it’s also a useful tool for those working in ops or infrastructure. This is because Node.js is a great tool for developing other development tools. If you’re someone working to support a team of developers, or, indeed, to help manage an entire distributed software infrastructure, it could be vital in empowering you to get creative and build your own support tools. Even more surprisingly, Node.js can be used in some IoT projects. As this post from 2016 suggests, the two things might not be quite such strange bedfellows. Node.js is a robust project that won't be going anywhere As I’ve already said, in the JavaScript world frameworks and tools can appear and disappear quickly. That means deciding what to learn, and, indeed, what to integrate into your stack, can feel like a bit of a gamble. However, you can be sure that Node.js is here to stay. There are a number of reasons for this. For starters, there’s no other tool that brings JavaScript to the server. But more than that, with Google betting heavily on V8 - which is, as we’ve seen, such an important part of the project - you can be sure it’s only going to go from strength to strength. It’s also worth pointing out that Node.js went through a small crisis when io.js broke away from the main Node.js project. This feud was as much personal as it was technical, but with the rift healed, and the Node.js Foundation now managing the whole project, helping to ensure that the software is continually evolving with other relevant technological changes and that the needs of the developers who use it continue to be met. Conclusion: spend some time exploring Node.js before you begin using it at work That’s just 5 reasons why you should learn Node.js. You could find more, but broadly speaking these all underline its importance in today’s development world. If you’re still not convinced, there’s a caveat. If Node.js isn’t yet right for you, don’t assume that it’s going to fix any technological or cultural issues that have been causing you headaches. It probably won’t. In fact, you should probably tackle those challenges before deciding to use it. But that all being said, even if you don’t think it’s the right time to use Node.js professionally, that doesn’t mean it isn’t worth learning. As you can see, it’s well worth your time. Who knows where it might take you? Ready to begin learning? Purchase Node.js Complete Reference Guide or read it for free with a subscription free trial.

In this tutorial, we will look at the installation process of TypeScript and the editor setup for TypeScript development. Microsoft does well in providing easy-to-perform steps to install TypeScript on all platforms, namely Windows, macOS, and Linux. [box type="shadow" align="" class="" width=""]The following excerpt is taken from the book TypeScript 2.x By Example written by Sachin Ohri. This book presents hands-on examples and projects to learn the fundamental concepts of the popular TypeScript programming language.[/box] Installation of TypeScript TypeScript's official website is the best source to install the latest version. On the website, go to the Download section. There, you will find details on how to install TypeScript. Node.js and Visual Studio are the two most common ways to get it. It supports a host of other editors and has plugins available for them in the same link. We will be installing TypeScript using Node.js and using Visual Studio Code as our primary editor. You can use any editor of your choice and be able to run the applications seamlessly. If you use full-blown Visual Studio as your primary development IDE, then you can use either of the links, Visual Studio 2017 or Visual Studio 2013, to download the TypeScript SDK. Visual Studio does come with a TypeScript compiler but it's better to install it from this link so as to get the latest version. To install TypeScript using Node.js, we will use npm (node package manager), which comes with Node.js. Node.js is a popular JavaScript runtime for building and running server-side JavaScript applications. As TypeScript compiles into JavaScript, Node is an ideal fit for developing server-side applications with the TypeScript language. As mentioned on the website, just running the following command in the Terminal (on macOS) / Command Prompt (on Windows) window will install the latest version: npm install -g typescript To load any package from Node.js, the npm command starts with npm install; the -g flag identifies that we are installing the package globally. The last parameter is the name of the package that we are installing. Once it is installed, you can check the version of TypeScript by running the following command in the Terminal window: tsc -v You can use the following command to get the help for all the other options that are available with tsc: tsc -h TypeScript editors One of the outstanding features of TypeScript is its support for editors. All the editors provide support for language services, thereby providing features such as IntelliSense, statement completion, and error highlighting. If you are coming from a .NET background, then Visual Studio 2013/2015/2017 is a good option for you. Visual Studio does not require any configuration and it's easy to start using TypeScript. As we discussed earlier, just install the SDK and you are good to go. If you are from a Java background, TypeScript supports Eclipse as well. It also supports plugins for Sublime, WebStorm, and Atom, and each of these provides a rich set of features. Visual Studio Code (VS Code) is another good option for an IDE. It's a smaller, lighter version of Visual Studio and primarily used for web application development. VS Code is lightweight and cross-platform, capable of running on Windows, Linux, and macOS. It has an ever-increasing set of plugins to help you write better code, such as TSLint, a static analysis tool to help TypeScript code for readability, maintainability, and error checking. VS Code has a compelling case to be the default IDE for all sorts of web application development. In this post, we will briefly look at the Visual Studio and VS Code setup for TypeScript. Visual Studio Visual Studio is a full-blown IDE provided by Microsoft for all .NET based development, but now Visual Studio also has excellent support for TypeScript with built-in project templates. A TypeScript compiler is integrated into Visual Studio to allow automatic transpiling of code to JavaScript. Visual Studio also has the TypeScript language service integrated to provide IntelliSense and design-time error checking, among other things. With Visual Studio, creating a project with a TypeScript file is as simple as adding a new file with a .ts extension. Visual Studio will provide all the features out of the box. VS Code VS Code is a lightweight IDE from Microsoft used for web application development. VS Code can be installed on Windows, macOS, and Linux-based systems. VS Code can recognize the different type of code files and comes with a huge set of extensions to help in development. You can install VS Code from https://code.visualstudio.com/download. VS Code comes with an integrated TypeScript compiler, so we can start creating projects directly. The following screenshot shows a TypeScript file opened in VS Code: To run the project in VS Code, we need a task runner. VS Code includes multiple task runners which can be configured for the project, such as Gulp, Grunt, and TypeScript. We will be using the TypeScript task runner for our build. VS Code has a Command Palette which allows you to access various different features, such as Build Task, Themes, Debug options, and so on. To open the Command Palette, use Ctrl + Shift + P on a Windows machine or Cmd + Shift + P on a macOS. In the Command Palette, type Build, as shown in the following screenshot, which will show the command to build the project: When the command is selected, VS Code shows an alert, No built task defined..., as follows: We select Configure Build Task and, from all the available options as shown in the following screenshot, choose TypeScript build: This creates a new folder in your project, .vscode and a new file, task.json. This JSON file is used to create the task that will be responsible for compiling TypeScript code in VS Code. TypeScript needs another JSON file (tsconfig.json) to be able to configure compiler options. Every time we run the code, tsc will look for a file with this name and use this file to configure itself. TypeScript is extremely flexible in transpiling the code to JavaScript as per developer requirements, and this is achieved by configuring the compiler options of TypeScript. TypeScript compiler The TypeScript compiler is called tsc and is responsible for transpiling the TypeScript code to JavaScript. The TypeScript compiler is also cross-platform and supported on Windows, macOS, and Linux. To run the TypeScript compiler, there are a couple of options. One is to integrate the compiler in your editor of choice, which we explained in the previous section. In the previous section, we also integrated the TypeScript compiler with VS Code, which allowed us to build our code from the editor itself. All the compiler configurations that we would want to use are added to the tsconfig.json file. Another option is to use tsc directly from the command line / Terminal window. TypeScript's tsc command takes compiler configuration options as parameters and compiles code into JavaScript. For example, create a simple TypeScript file in Notepad and add the following lines of code to it. To create a file as a TypeScript file, we just need to make sure we have the file extension as *.ts: class Editor { constructor(public name: string,public isTypeScriptCompatible : Boolean) {} details() { console.log('Editor: ' + this.name + ', TypeScript installed: ' + this.isTypeScriptCompatible); } } class VisualStudioCode extends Editor{ public OSType: string constructor(name: string,isTypeScriptCompatible : Boolean, OSType: string) { super(name,isTypeScriptCompatible); this.OSType = OSType; } } let VS = new VisualStudioCode('VSCode', true, 'all'); VS.details(); This is the same code example we used in the TypeScript features section of this chapter. Save this file as app.ts (you can give it any name you want, as long as the extension of the file is *.ts). In the command line / Terminal window, navigate to the path where you have saved this file and run the following command: tsc app.ts This command will build the code and the transpile it into JavaScript. The JavaScript file is also saved in the same location where we had TypeScript. If there is any build issue, tsc will show these messages on the command line only. As you can imagine, running the tsc command manually for medium- to large-scale projects is not a productive approach. Hence, we prefer to use an editor that has TypeScript integrated. The following table shows the most commonly used TypeScript compiler configurations. We will be discussing these in detail in upcoming chapters: Compiler option Type Description allowUnusedLabels boolean By default, this flag is false. This option tells the compiler to flag unused labels. alwaysStrict boolean By default, this flag is false. When turned on, this will cause the compiler to compile in strict mode and emit use strict in the source file. module string Specify module code generation: None, CommonJS, AMD, System, UMD, ES6, or ES2015. moduleResolution string Determines how the module is resolved. noImplicitAny boolean This property allows an error to be raised if there is any code which implies data type as any. This flag is recommended to be turned off if you are migrating a JavaScript project to TypeScript in an incremental manner. noImplicitReturn boolean Default value is false; raises an error if not all code paths return a value. noUnusedLocals boolean Reports an error if there are any unused locals in the code. noUnusedParameter boolean Reports an error if there are any unused parameters in the code. outDir string Redirects output structure to the directory. outFile string Concatenates and emits output to a single file. The order of concatenation is determined by the list of files passed to the compiler on the command line along with triple-slash references and imports. See the output file order documentation for more details. removeComments boolean Remove all comments except copyright header comments beginning with /*!. sourcemap boolean Generates corresponding .map file. Target string Specifies ECMAScript target version: ES3(default), ES5, ES6/ES2015, ES2016, ES2017, or ESNext. Watch Runs the compiler in watch mode. Watches input files and triggers recompilation on changes. We saw it is quite easy to set up and configure TypeScript, and we are now ready to get started with our first application! To learn more about writing and compiling your first TypeScript application, make sure you check out the book TypeScript 2.x By Example. Introduction to TypeScript Introducing Object Oriented Programmng with TypeScript Elm and TypeScript – Static typing on the Frontend

Whether you’re new to software engineering or have years of experience under your belt, knowing what to learn can be difficult. Which programming language should you learn first? Which programming language should you learn next? There are hundreds of programming languages in widespread use. Each one has different features, many even have very different applications. Of course, many do not too - and knowing which one to use can be tough. Making those decisions is as important a part of being a developer as it is writing code. Just as English is the international language for most businesses and French is the language of love, different programming languages are better suited for different purposes. Let us take a look at what developers have chosen to be the top programming languages for 2018 in this year’s Packt Skill Up Survey. Source: Packt Skill Up Survey 2018 Java reigns supreme Java, the accessible and ever-present programming language continues to be widespread year on year. It is a 22 year old language which if put into human perspective is incredible. You would have been old enough to have finished college, have a celebratory alcoholic drink, gamble in Iowa and get married without parental consent in Mississippi! With time and age, Java has proven its consistency as a reliable programming language for engineers and developers. Our Skill Up 2018 survey data reveals it’s still the most popular programming language. Perhaps one of the reasons for this is Java’s flexibility and adaptability. Programs can run on several different types of machines; as long as the computer has a Java Runtime Environment (JRE) installed, a Java program can run on it. Most types of computers will be compatible with a JRE. PCs running on Windows, Macs, Unix or Linux computers, even large mainframe computers and mobile phones will be compatible with Java. Another reason for Java’s popularity is because it is object oriented. Java code is robust because Java objects contain no references to data external to themselves. This is why Java is so widely used across industry. It’s reliable and secure. When it's time for mobile developers to build Android apps, the first and most popular option is Java. Java is the official language for Android development which means it has great support from Google and most apps on Playstore are built on Java. If one talks about job opportunities in field of Java, there are plenty such as ‘Java-UI Developers’, ’Android Developers’ and many others. Hence, there are numerous jobs opportunities available in Java, J2EE combining with other new technologies. These technologies are among the highest paid jobs in IT industry today. According to Payscale, an average Java Developer salary in the USA is around $102,000 with salaries for job postings nationwide being 77% higher than average salaries. Some of the widely known domains where Java is used extensively is financial services, banking, stock market, retail and scientific and research communities. Finally, it’s worth noting that the demand for Java developers is pretty high and given it’s the language required in many engineering roles. It does make sense to start learning it if you don’t yet know it. Start learning Java. Explore Packt’s latest Java eBooks and videos Java Tutorials: Design a RESTful web API with Java JavaScript retains the runner up spot JavaScript for years kept featuring in the list of top programming languages and this time it is 2nd after Java. JavaScript has continued to be everywhere from front end web pages to mobile web apps and everything in between. A web developer can add personality to websites by using JavaScript. It is the native language of the browser. If you want to build single-page web apps, there is really only one language option for building client-side single-page apps, and that is JavaScript. It is supported by all popular browsers like Microsoft Internet Explorer (beginning with version 3.0), Firefox, Safari, Opera, Google Chrome, etc. JavaScript has been the most versatile and popular among developers because, it is simple to learn, gives extended functionality to web pages, is an inexpensive language. In other words, it does not require any special compilers or text editors to run the script. And it is simple to implement and is relatively fast for end users. After the release of Node.js in 2009, the "JavaScript everywhere" paradigm has become a reality. This server-side JavaScript framework allows to unify web application development around a single programming language, rather than rely on a different language. npm, Node.js's package manager, is the largest ecosystem of open source libraries in the world. Node.js is also being adopted in IoT projects due to its speed, number of plugins and scalability convenience. Another open-source JavaScript library known as React Native lets you build cross platform mobile applications in order to make a smooth transition from web to mobile. According to Daxx, JavaScript developers grab some of the highest paid tech salaries with an average of $96,000 in the US. There are plenty more sources that name JavaScript as one of the most sought-after skills in 2017. ITJobsWatch ranked JavaScript as the second most in-demand programming language in the UK, a conclusion based on the number of job ads posted over the last three months. Read More: 5 JavaScript Machine learning libraries you need to know Python on the rise Python is a general-purpose language; often described as utilitarian, a word which makes it sound a little boring. It is in fact anything but that. Why is it considered among the top programming languages? Simple: it is a truly universal language, applicable to a range of problems and areas. If programmers begin working with Python for one job or career, they can easily jump to another, even if it’s in an unrelated industry. Google uses Python in a number of applications. Additionally it has a developer portal devoted to Python. It features free classes so employees can learn more about the language. Here are just a few reasons why Python is so popular: Python comes with plenty of documentation, guides, tutorials and a developer community which is incredibly active for timely help and support. Python has Google as one of its corporate sponsors. Google contributes to the growing list of documentation and support tools, and effectively acts as a high profile evangelist for Python. Python is one of the most popular languages for data science. It’s used to build machine learning and AI systems. It comes with excellent sets of task specific libraries from Numpy and Scipy for scientific computing to Django for web development. Ask any Python developer and they have to agree Python is speedy, reliable and efficient. You can deploy Python applications on any environment and there is little to no performance loss no matter which platform. Python is easy to learn probably because it lets you think like programmer, as it is easily readable and almost looks like everyday English. The learning curve is very gradual than all other programming languages which tend to be quite steep. Python contains everything from data structures, tools, support, Python community, the Python Software Foundation and everything combined makes it a favorite among the developers. According to Gooroo, a platform that provides tech skill and salary analytics, Python is one of the highest-paying programming languages in the USA. In fact, at $103,492 per year, Python developers are on an average the second best-paid in the country. The language is used for system operations, web development, server and administrative tools, deployment, scientific modeling, and now in building AI and machine learning systems among others. Read More: What are professionals planning to learn this year? Python, deep learning, yes. But also... C# is not left behind Since the introduction of C# in 2002 by Microsoft, C#’s popularity has remained relatively high. It is a general-purpose language designed for developing apps on the Microsoft platform. C# can be used to create almost anything but it’s particularly strong at building Windows desktop applications and games. It can also be used to develop web applications and has become increasingly popular for mobile development too. Cross-platform tools such as Xamarin allow apps written in C# to be used on almost any mobile device. C# is widely used to create games using the Unity game engine, which is the most popular game engine today. Although C#’s syntax is more logical and consistent than C++, it is a complex language. Mastering it may take more time than languages like Python. The average pay for a C# Developer according to Payscale is $69,006 per year. With C# you have solid prospects as big finance corporations are using C# as their choice of language. In the News: Exciting New Features in C# 8.0 C# Tutorial: Behavior Scripting in C# and Javascript for game developers SQL remains strong The acronym SQL stands for Structured Query Language. This essentially means “a programming language that is used to communicate with database in a structured way”. Much like how Javascript is necessary for making websites exciting and more than just a static page, SQL is one of the only two languages widely used to communicate with databases. SQL is one of the very few languages where you describe what you want, not how to get it.  That means the database can be much more intelligent about how it decides to build its response, and is robust to changes in the computing environment it runs on. It is about a set theory which forces you to think very clearly about what it is you want, and express that in a precise way. More importantly, it gives you a vocabulary and set of tools for thinking about the problem you’re trying to solve without reference to the specific idioms of your application. SQL is a critical skill for many data-related roles. Data scientists and analysts will need to know SQL, for example. But as data reaches different parts of modern business, such as marketing and product management, it’s becoming a useful language for non-developers as well to learn. Source: Packt Skill Up Survey 2018 By far MySQL is still the most commonly used databases for web based applications in 2018, according to the report. It’s freeware, but it is frequently updated with features and security improvements. It offers a lot of functionality even for a free database engine. There are a variety of user interfaces that can be implemented. It can be made to work with other databases, including DB2 and Oracle. Organizations that need a robust database management tool but are on a budget, MySQL will be a perfect choice for them. The average salary for a Senior SQL Developer according to Glassdoor is $100,271 in the US. Unlike other areas in IT industry, job options and growth criteria for SQL developer is completely different. You may work as a database administrator, system manager, SQL professionals etc it completely depends on the functional knowledge and experience you have gained. Read More: 12 most common MySQL errors you should be aware of C++ also among favorites C++, the general purpose language for systems programming, was designed in 1979 by Bjarne Stroustrup. Despite competition from other powerful languages such as Java, Python, and SQL, it is still prevalent among developers. People have been predicting its demise for more than 20 years, but it's still growing. Basically, nothing that can handle complexity runs as fast as C++. C++ is designed for fairly hardcore applications, and it's generally used together with some scripting language or other. It is for building systems that need high performance, high reliability, small footprint, low energy consumption, all of these good things. That’s why you see telecom and financial applications built on C++. C++ is also considered to be one of the best solutions for creating applications that process music and film. There is even an extensive list of websites and tools created based on C++. Financial analysts predict that earnings in this specialization will reach at least $102,000 per year. C++ Tutorials: Getting started with C++ Features Read More: Working with Shaders in C++ to create 3D games Other programming languages like C, PHP, Swift, Go etc were also on the developer’s choice list. The creation of supporting technologies in the last few years has given rise to speculation that programming languages are converging in terms of their capabilities. For example, Node.js enables Javascript developers to create back-end functionalities - relinquishing the need to learn PHP or even hiring a separate back-end developer altogether. As of now, we can only speculate on future tech trends. However, definitely it will be worth keeping an eye out for which horse to bet on! 20 ways to describe programming in 5 words A really basic guide to batch file programming What is Mob Programming?

As a technology research analyst, I try to keep up the pace with the changing world of technology. It seems like every single day, there is a new programming language, framework, or tool emerging out of nowhere. In order to keep up, I regularly have a peek at the listicles on TIOBE, PyPL, and Stackoverflow along with some twitter handles and popular blogs, which keeps my FOMO (fear of missing out) in check. So here I was, strolling through the TIOBE index, to see if a new programming language is making the rounds or if any old timer language is facing its doomsday in the lower half of the table. The first thing that caught my attention was Python, which interestingly broke into the top 3 for the first time since it was ranked by TIOBE. I never cared to look at Java, since it has been claiming the throne ever since it became popular. But with my pupils dilated, I saw something which I would have never expected, especially with the likes of Python, C#, Swift, and JavaScript around. There it was, the language which everyone seemed to have forgotten about, C, sitting at the second position, like an old tower among the modern skyscrapers in New York. A quick scroll down shocked me even more: C was only recently named the language of 2017 by TIOBE. The reason it won was because of its impressive yearly growth of 1.69% and its consistency - C has been featured in the top 3 list for almost four decades now. This result was in stark contrast to many news sources (including Packt’s own research) that regularly place languages like Python and JavaScript on top of their polls. But surely this was an indicator of something. Why would a language which is almost 50 years old still hold its ground against the ranks of newer programming language? C has a design philosophy for the ages A solution to the challenges of UNIX and Assembly The 70s was a historic decade for computing. Many notable inventions and developments, particularly in the area of networking, programming, and file systems, took place. UNIX was one such revolutionary milestone, but the biggest problem with UNIX was that it was programmed in Assembly language. Assembly was fine for machines, but difficult for humans. Watch now: Learn and Master C Programming For Absolute Beginners So, the team working on UNIX, namely Dennis Ritchie, Ken Thompson, and Brian Kernighan decided to develop a language which could understand data types and supported data structures. They wanted C to be as fast as the Assembly but with the features of a high-level language. And that’s how C came into existence, almost out of necessity. But the principles on which the C programming language was built were not coincidental. It compelled the programmers to write better code and strive for efficiency rather than being productive by providing a lot of abstractions. Let’s discuss some features which makes C a language to behold. Portability leads to true ubiquity When you try to search for the biggest feature of C, almost instantly, you are bombarded with articles on portability. Which makes you wonder what is it about portability that makes C relevant in the modern world of computing. Well, portability can be defined as the measure of how easily software can be transferred from one computer environment or architecture to another. One can also argue that portability is directly proportional to how flexible your software is. Applications or software developed using C are considered to be extremely flexible because you can find a C compiler for almost every possible platform available today. So if you develop your application by simply exercising some discipline to write portable code, you have yourself an application which virtually runs on every major platform. Programmer-driven memory management It is universally accepted that C is a high-performance language. The primary reason for this is that it works very close to the machine, almost like an Assembly language. But very few people realize that versatile features like explicit memory management makes C one of the better-performing languages out there. Memory management allows programmers to scale down a program to run with a small amount of memory. This feature was important in the early days because the computers or terminals as they used to call it, were not as powerful as they are today. But the advent of mobile devices and embedded systems has renewed the interest of programmers in C language because these mobile devices demand that the programmers keep memory requirement to a minimum. Many of the programming languages today provide functionalities like garbage collection that takes care of the memory allocation. But C calls programmers’ bluff by asking them to be very specific. This makes their programs and its memory efficient and inherently fast. Manual memory management makes C one of the most suitable languages for developing other programming languages. This is because even in a garbage collector someone has to take care of memory allocation - that infrastructure is provided by C. Structure is all I got As discussed before, Assembly was difficult to work with, particularly when dealing with large chunks of code. C has a structured approach in its design which allows the programmers to break down the program into multiple blocks of code for execution, often called as procedures or functions. There are, of course, multiple ways in which software development can be approached. Structural programming is one such approach that is effective when you need to break down a problem into its component pieces and then convert it into application code. Although it might not be quite as in vogue as object-oriented programming is today, this approach is well suited to tasks like database scripting or developing small programs with logical sequences to carry out specific set of tasks. As one of the best languages for structural programming, it’s easy to see how C has remained popular, especially in the context of embedded systems and kernel development. Applications that stand the test of time If Beyoncé would have been a programmer, she definitely might have sang “Who runs the world? C developers”. And she would have been right. If you’re using a digital alarm clock, a microwave, or a car with anti-lock brakes, chances are that they have been programmed using C. Though it was never developed specifically for embedded systems, C has become the defacto programming language for embedded developers, systems programmers, and kernel development. C: the backbone of our operating systems We already know that the world famous UNIX system was developed in C, but is it the only popular application that has been developed using C? You’ll be astonished to see the list of applications that follows: The world desktop operating market is dominated by three major operating systems: Windows, MAC, and Linux. The kernel of all these OSes has been developed using the C programming language. Similarly, Android, iOS, and Windows are some of the popular mobile operating systems whose kernels were developed in C. Just like UNIX, the development of Oracle Database began on Assembly and then switched to C. It’s still widely regarded as one of the best database systems in the world. Not only Oracle but MySQL and PostgreSQL have also been developed using C - the list goes on and on. What does the future hold for C? So far we discussed the high points of C programming, it’s design principle and the applications that were developed using it. But the bigger question to ask is, what its future might hold. The answer to this question is tricky, but there are several indicators which show positive signs. IoT is one such domain where the C programming language shines. Whether or not beginner programmers should learn C has been a topic of debate everywhere. The general consensus says that learning C is always a good thing, as it builds up your fundamental knowledge of programming and it looks good on the resume. But IoT provides another reason to learn C, due to the rapid growth in the IoT industry. We already saw the massive number of applications built on C and their codebase is still maintained in it. Switching to a different language means increased cost for the company. Since it is used by numerous enterprises across the globe the demand for C programmers is unlikely to vanish anytime soon. Read Next Rust as a Game Programming Language: Is it any good? Google releases Oboe, a C++ library to build high-performance Android audio apps Will Rust Replace C++?

In the earlier days, many software solution providers did not really pay attention to documenting their RESTful web APIs. However, many API vendors soon realized the need for a good API documentation solution. Today, you will find a variety of approaches to documenting RESTful web APIs. There are some popular solutions available today for describing, producing, consuming, and visualizing RESTful web services. In this tutorial, we will explore Swagger which offers a specification and a complete framework implementation for describing, producing, consuming, and visualizing RESTful web services. The Swagger framework works with many popular programming languages, such as Java, Scala, Clojure, Groovy, JavaScript, and .NET. This tutorial is an extract taken from the book RESTFul Java Web Services - Third Edition, written by Bogunuva Mohanram Balachandar. This book will help you master core REST concepts and create RESTful web services in Java. A glance at the market adoption of Swagger The greatest strength of Swagger is its powerful API platform, which satisfies the client, documentation, and server needs. The Swagger UI framework serves as the documentation and testing utility. Its support for different languages and its matured tooling support have really grabbed the attention of many API vendors, and it seems to be the one with the most traction in the community today. Swagger is built using Scala. This means that when you package your application, you need to have the entire Scala runtime into your build, which may considerably increase the size of your deployable artifact (the EAR or WAR file). That said, Swagger is however improving with each release. For example, the Swagger 2.0 release allows you to use YAML for describing APIs. So, keep a watch on this framework. The Swagger framework has the following three major components: Server: This component hosts the RESTful web API descriptions for the services that the clients want to use Client: This component uses the RESTful web API descriptions from the server to provide an automated interfacing mechanism to invoke the REST APIs User interface: This part of the framework reads a description of the APIs from the server, renders it as a web page, and provides an interactive sandbox to test the APIs A quick overview of the Swagger structure Let's take a quick look at the Swagger file structure before moving further. The Swagger 1.x file contents that describe the RESTful APIs are represented as the JSON objects. With Swagger 2.0 release onwards, you can also use the YAML format to describe the RESTful web APIs. This section discusses the Swagger file contents represented as JSON. The basic constructs that we'll discuss in this section for JSON are also applicable for the YAML representation of APIs, although the syntax differs. When using the JSON structure for describing the REST APIs, the Swagger file uses a Swagger object as the root document object for describing the APIs. Here is a quick overview of the various properties that you will find in a Swagger object: The following properties describe the basic information about the RESTful web application: swagger: This property specifies the Swagger version. info: This property provides metadata about the API. host: This property locates the server where the APIs are hosted. basePath: This property is the base path for the API. It is relative to the value set for the host field. schemes: This property transfers the protocol for a RESTful web API, such as HTTP and HTTPS. The following properties allow you to specify the default values at the application level, which can be optionally overridden for each operation: consumes: This property specifies the default internet media types that the APIs consume. It can be overridden at the API level. produces: This property specifies the default internet media types that the APIs produce. It can be overridden at the API level. securityDefinitions: This property globally defines the security schemes for the document. These definitions can be referred to from the security schemes specified for each API. The following properties describe the operations (REST APIs): paths: This property specifies the path to the API or the resources. The path must be appended to the basePath property in order to get the full URI. definitions: This property specifies the input and output entity types for operations on REST resources (APIs). parameters: This property specifies the parameter details for an operation. responses: This property specifies the response type for an operation. security: This property specifies the security schemes in order to execute this operation. externalDocs: This property links to the external documentation. A complete Swagger 2.0 specification is available at Github. The Swagger specification was donated to the Open API initiative, which aims to standardize the format of the API specification and bring uniformity in usage. Swagger 2.0 was enhanced as part of the Open API initiative, and a new specification OAS 3.0 (Open API specification 3.0) was released in July 2017. The tools are still being worked on to support OAS3.0. I encourage you to explore the Open API Specification 3.0 available at Github. Overview of Swagger APIs The Swagger framework consists of many sub-projects in the Git repository, each built with a specific purpose. Here is a quick summary of the key projects: swagger-spec: This repository contains the Swagger specification and the project in general. swagger-ui: This project is an interactive tool to display and execute the Swagger specification files. It is based on swagger-js (the JavaScript library). swagger-editor: This project allows you to edit the YAML files. It is released as a part of Swagger 2.0. swagger-core: This project provides the scala and java library to generate the Swagger specifications directly from code. It supports JAX-RS, the Servlet APIs, and the Play framework. swagger-codegen: This project provides a tool that can read the Swagger specification files and generate the client and server code that consumes and produces the specifications. In the next section, you will learn how to use the swagger-core project offerings to generate the Swagger file for a JAX-RS application. Generating Swagger from JAX-RS Both the WADL and RAML tools that we discussed in the previous sections use the JAX-RS annotations metadata to generate the documentation for the APIs. The Swagger framework does not fully rely on the JAX-RS annotations but offers a set of proprietary annotations for describing the resources. This helps in the following scenarios: The Swagger core annotations provide more flexibility for generating documentations compliant with the Swagger specifications It allows you to use Swagger for generating documentations for web components that do not use the JAX-RS annotations, such as servlet and the servlet filter The Swagger annotations are designed to work with JAX-RS, improving the quality of the API documentation generated by the framework. Note that the swagger-core project is currently supported on the Jersey and Restlet implementations. If you are considering any other runtime for your JAX-RS application, check the respective product manual and ensure the support before you start using Swagger for describing APIs. Some of the commonly used Swagger annotations are as follows: The @com.wordnik.swagger.annotations.Api annotation marks a class as a Swagger resource. Note that only classes that are annotated with @Api will be considered for generating the documentation. The @Api annotation is used along with class-level JAX-RS annotations such as @Produces and @Path. Annotations that declare an operation are as follows: @com.wordnik.swagger.annotations.ApiOperation: This annotation describes a resource method (operation) that is designated to respond to HTTP action methods, such as GET, PUT, POST, and DELETE. @com.wordnik.swagger.annotations.ApiParam: This annotation is used for describing parameters used in an operation. This is designed for use in conjunction with the JAX-RS parameters, such as @Path, @PathParam, @QueryParam, @HeaderParam, @FormParam, and @BeanParam. @com.wordnik.swagger.annotations.ApiImplicitParam: This annotation allows you to define the operation parameters manually. You can use this to override the @PathParam or @QueryParam values specified on a resource method with custom values. If you have multiple ImplicitParam for an operation, wrap them with @ApiImplicitParams. @com.wordnik.swagger.annotations.ApiResponse: This annotation describes the status codes returned by an operation. If you have multiple responses, wrap them by using @ApiResponses. @com.wordnik.swagger.annotations.ResponseHeader: This annotation describes a header that can be provided as part of the response. @com.wordnik.swagger.annotations.Authorization: This annotation is used within either Api or ApiOperation to describe the authorization scheme used on a resource or an operation. Annotations that declare API models are as follows: @com.wordnik.swagger.annotations.ApiModel: This annotation describes the model objects used in the application. @com.wordnik.swagger.annotations.ApiModelProperty: This annotation describes the properties present in the ApiModel object. A complete list of the Swagger core annotations is available at Github. Having learned the basics of Swagger, it is time for us to move on and build a simple example to get a feel of the real-life use of Swagger in a JAX-RS application. As always, this example uses the Jersey implementation of JAX-RS. Specifying dependency to Swagger To use Swagger in your Jersey 2 application, specify the dependency to swagger-jersey2-jaxrs jar. If you use Maven for building the source, the dependency to the swagger-core library will look as follows: <dependency> <groupId>com.wordnik</groupId> <artifactId>swagger-jersey2-jaxrs</artifactId> <version>1.5.1-M1</version> <!-use the appropriate version here, 1.5.x supports Swagger 2.0 spec --> </dependency> You should be careful while choosing the swagger-core version for your product. Note that swagger-core 1.3 produces the Swagger 1.2 definitions, whereas swagger-core 1.5 produces the Swagger 2.0 definitions. The next step is to hook the Swagger provider components into your Jersey application. This is done by configuring the Jersey servlet (org.glassfish.jersey.servlet.ServletContainer) in web.xml, as shown here: <servlet> <servlet-name>jersey</servlet-name> <servlet-class> org.glassfish.jersey.servlet.ServletContainer </servlet-class> <init-param> <param-name>jersey.config.server.provider.packages </param-name> <param-value> com.wordnik.swagger.jaxrs.json, com.packtpub.rest.ch7.swagger </param-value> </init-param> <load-on-startup>1</load-on-startup> </servlet> <servlet-mapping> <servlet-name>jersey</servlet-name> <url-pattern>/webresources/*</url-pattern> </servlet-mapping> To enable the Swagger documentation features, it is necessary to load the Swagger framework provider classes from the com.wordnik.swagger.jaxrs.listing package. The package names of the JAX-RS resource classes and provider components are configured as the value for the jersey.config.server.provider.packages init parameter. The Jersey framework scans through the configured packages for identifying the resource classes and provider components during the deployment of the application. Map the Jersey servlet to a request URI so that it responds to the REST resource calls that match the URI. If you prefer not to use web.xml, you can also use the custom application subclass for (programmatically) specifying all the configuration entries discussed here. To try this option, refer to Github. Configuring the Swagger definition After specifying the Swagger provider components, the next step is to configure and initialize the Swagger definition. This is done by configuring the com.wordnik.swagger.jersey.config.JerseyJaxrsConfig servlet in web.xml, as follows: <servlet> <servlet-name>Jersey2Config</servlet-name> <servlet-class> com.wordnik.swagger.jersey.config.JerseyJaxrsConfig </servlet-class> <init-param> <param-name>api.version</param-name> <param-value>1.0.0</param-value> </init-param> <init-param> <param-name>swagger.api.basepath</param-name> <param-value> http://localhost:8080/hrapp/webresources </param-value> </init-param> <load-on-startup>1</load-on-startup> </servlet> Here is a brief overview of the initialization parameters used for JerseyJaxrsConfig: api.version: This parameter specifies the API version for your application swagger.api.basepath: This parameter specifies the base path for your application With this step, we have finished all the configuration entries for using Swagger in a JAX-RS (Jersey 2 implementation) application. In the next section, we will see how to use the Swagger metadata annotation on a JAX-RS resource class for describing the resources and operations. Adding a Swagger annotation on a JAX-RS resource class Let's revisit the DepartmentResource class used in the previous sections. In this example, we will enhance the DepartmentResource class by adding the Swagger annotations discussed earlier. We use @Api to mark DepartmentResource as the Swagger resource. The @ApiOperation annotation describes the operation exposed by the DepartmentResource class: import com.wordnik.swagger.annotations.Api; import com.wordnik.swagger.annotations.ApiOperation; import com.wordnik.swagger.annotations.ApiParam; import com.wordnik.swagger.annotations.ApiResponse; import com.wordnik.swagger.annotations.ApiResponses; //Other imports are removed for brevity @Stateless @Path("departments") @Api(value = "/departments", description = "Get departments details") public class DepartmentResource { @ApiOperation(value = "Find department by id", notes = "Specify a valid department id", response = Department.class) @ApiResponses(value = { @ApiResponse(code = 400, message = "Invalid department id supplied"), @ApiResponse(code = 404, message = "Department not found") }) @GET @Path("{id}") @Produces("application/json") public Department findDepartment( @ApiParam(value = "The department id", required = true) @PathParam("id") Integer id){ return findDepartmentEntity(id); } //Rest of the codes are removed for brevity } To view the Swagger documentation, build the source and deploy it to the server. Once the application is deployed, you can navigate to http://<host>:<port>/<application-name>/<application-path>/swagger.json to view the Swagger resource listing in the JSON format. The Swagger URL for this example will look like the following: http://localhost:8080/hrapp/webresource/swagger.json The following sample Swagger representation is for the DepartmentResource class discussed in this section: { "swagger": "2.0", "info": { "version": "1.0.0", "title": "" }, "host": "localhost:8080", "basePath": "/hrapp/webresources", "tags": [ { "name": "user" } ], "schemes": [ "http" ], "paths": { "/departments/{id}": { "get": { "tags": [ "user" ], "summary": "Find department by id", "description": "", "operationId": "loginUser", "produces": [ "application/json" ], "parameters": [ { "name": "id", "in": "path", "description": "The department id", "required": true, "type": "integer", "format": "int32" } ], "responses": { "200": { "description": "successful operation", "schema": { "$ref": "#/definitions/Department" } }, "400": { "description": "Invalid department id supplied" }, "404": { "description": "Department not found" } } } } }, "definitions": { "Department": { "properties": { "departmentId": { "type": "integer", "format": "int32" }, "departmentName": { "type": "string" }, "_persistence_shouldRefreshFetchGroup": { "type": "boolean" } } } } } As mentioned at the beginning of this section, from the Swagger 2.0 release onward it supports the YAML representation of APIs. You can access the YAML representation by navigating to swagger.yaml. For instance, in the preceding example, the following URI gives you the YAML file: http://<host>:<port>/<application-name>/<application-path>/swagger.yaml The complete source code for this example is available at the Packt website. You can download the example from the Packt website link that we mentioned at the beginning of this book, in the Preface section. In the downloaded source code, see the rest-chapter7-service-doctools/rest-chapter7-jaxrs2swagger project. Generating a Java client from Swagger The Swagger framework is packaged with the Swagger code generation tool as well (swagger-codegen-cli), which allows you to generate client libraries by parsing the Swagger documentation file. You can download the swagger-codegen-cli.jar file from the Maven central repository by searching for swagger-codegen-cli in search maven. Alternatively, you can clone the Git repository and build the source locally by executing mvn install. Once you have swagger-codegen-cli.jar locally available, run the following command to generate the Java client for the REST API described in Swagger: java -jar swagger-codegen-cli.jar generate -i <Input-URI-or-File-location-for-swagger.json> -l <client-language-to-generate> -o <output-directory> The following example illustrates the use of this tool: java -jar swagger-codegen-cli-2.1.0-M2.jar generate -i http://localhost:8080/hrapp/webresources/swagger.json -l java -o generated-sources/java When you run this tool, it scans through the RESTful web API description available at http://localhost:8080/hrapp/webresources/swagger.json and generates a Java client source in the generated-sources/java folder. Note that the Swagger code generation process uses the mustache templates for generating the client source. If you are not happy with the generated source, Swagger lets you specify your own mustache template files. Use the -t flag to specify your template folder. To learn more, refer to the README.md file at Github. To learn more grab this latest edition of RESTful Java Web Services to build robust, scalable and secure RESTful web services using Java APIs. Getting started with Django RESTful Web Services How to develop RESTful web services in Spring Testing RESTful Web Services with Postman

Odoo uses a client/server architecture in which clients are web browsers accessing the Odoo server via RPC. Both the server and client extensions are packaged as modules which are optionally loaded in a database. Odoo modules can either add brand new business logic to an Odoo system or alter and extend existing business logic. Everything in Odoo starts and ends with modules. In this article, we will cover the basics of creating Odoo Addon Modules. Recipes that we will cover in this. Creating and installing a new addon module Completing the addon module manifest Organizing the addon module file structure Adding Models Our main goal here is to understand how an addon module is structured and the typical incremental workflow to add components to it. This post is an excerpt from the book Odoo 11 Development Cookbook - Second Edition by Alexandre Fayolle and Holger Brunn. With this book, you can create fast and efficient server-side applications using the latest features of Odoo v11. For this article, you should have Odoo installed. You are also expected to be comfortable in discovering and installing extra addon modules. Creating and installing a new addon module In this recipe, we will create a new module, make it available in our Odoo instance, and install it. Getting ready We will need an Odoo instance ready to use. For explanation purposes, we will assume Odoo is available at ~/odoo-dev/odoo, although any other location of your preference can be used. We will also need a location for our Odoo modules. For the purpose of this recipe, we will use a local-addons directory alongside the odoo directory, at ~/odoo-dev/local-addons. How to do it... The following steps will create and install a new addon module: Change the working directory in which we will work and create the addons directory where our custom module will be placed: $ cd ~/odoo-dev $ mkdir local-addons Choose a technical name for the new module and create a directory with that name for the module. For our example, we will use my_module: $ mkdir local-addons/my_module A module's technical name must be a valid Python identifier; it must begin with a letter, and only contain letters (preferably lowercase), numbers, and underscore characters. Make the Python module importable by adding an __init__.py file: $ touch local-addons/my_module/__init__.py Add a minimal module manifest for Odoo to detect it. Create a __manifest__.py file with this line: {'name': 'My module'} Start your Odoo instance including our module directory in the addons path: $ odoo/odoo-bin --addons-path=odoo/addon/,local-addons/ If the --save option is added to the Odoo command, the addons path will be saved in the configuration file. Next time you start the server, if no addons path option is provided, this will be used. Make the new module available in your Odoo instance; log in to Odoo using admin, enable the Developer Mode in the About box, and in the Apps top menu, select Update Apps List. Now, Odoo should know about our Odoo module. Select the Apps menu at the top and, in the search bar in the top-right, delete the default Apps filter and search for my_module. Click on its Install button, and the installation will be concluded. How it works... An Odoo module is a directory containing code files and other assets. The directory name used is the module's technical name. The name key in the module manifest is its title. The __manifest__.py file is the module manifest. It contains a Python dictionary with information about the module, the modules it depends on, and the data files that it will load. In the example, a minimal manifest file was used, but in real modules, we will want to add a few other important keys. These are discussed in the Completing the module manifest recipe, which we will see next. The module directory must be Python-importable, so it also needs to have an __init__.py file, even if it's empty. To load a module, the Odoo server will import it. This will cause the code in the __init__.py file to be executed, so it works as an entry point to run the module Python code. Due to this, it will usually contain import statements to load the module Python files and submodules. Known modules can be installed directly from the command line using the --init or -i option. This list is initially set when you create a new database, from the modules found on the addons path provided at that time. It can be updated in an existing database with the Update Module List menu. Completing the addon module manifest The manifest is an important piece for Odoo modules. It contains important information about the module and declares the data files that should be loaded. Getting ready We should have a module to work with, already containing a __manifest__.py manifest file. You may want to follow the previous recipe to provide such a module to work with. How to do it... We will add a manifest file and an icon to our addon module: To create a manifest file with the most relevant keys, edit the module __manifest__.py file to look like this: { 'name': "Title", 'summary': "Short subtitle phrase", 'description': """Long description""", 'author': "Your name", 'license': "AGPL-3", 'website': "http://www.example.com", 'category': 'Uncategorized', 'version': '11.0.1.0.0', 'depends': ['base'], 'data': ['views.xml'], 'demo': ['demo.xml'], } To add an icon for the module, choose a PNG image to use and copy it to static/description/icon.png. How it works... The remaining content is a regular Python dictionary, with keys and values. The example manifest we used contains the most relevant keys: name: This is the title for the module. summary: This is the subtitle with a one-line description. description: This is a long description written in plain text or the ReStructuredText (RST) format. It is usually surrounded by triple quotes, and is used in Python to delimit multiline texts. For an RST quickstart reference, visit http://docutils.sourceforge.net/docs/user/rst/quickstart.html. author: This is a string with the name of the authors. When there is more than one, it is common practice to use a comma to separate their names, but note that it still should be a string, not a Python list. license: This is the identifier for the license under which the module is made available. It is limited to a predefined list, and the most frequent option is AGPL-3. Other possibilities include LGPL-3, Other OSI approved license, and Other proprietary. website: This is a URL people should visit to know more about the module or the authors. category: This is used to organize modules in areas of interest. The list of the standard category names available can be seen at https://github.com/odoo/odoo/blob/11.0/odoo/addons/base/module/module_data.xml. However, it's also possible to define other new category names here. version: This is the modules' version numbers. It can be used by the Odoo app store to detect newer versions for installed modules. If the version number does not begin with the Odoo target version (for example, 11.0), it will be automatically added. Nevertheless, it will be more informative if you explicitly state the Odoo target version, for example, using 11.0.1.0.0 or 11.0.1.0 instead of 1.0.0 or 1.0. depends: This is a list with the technical names of the modules it directly depends on. If none, we should at least depend on the base module. Don't forget to include any module defining XML IDs, Views, or Models referenced by this module. That will ensure that they all load in the correct order, avoiding hard-to-debug errors. data: This is a list of relative paths to the data files to load with module installation or upgrade. The paths are relative to the module root directory. Usually, these are XML and CSV files, but it's also possible to have YAML data files. demo: This is the list of relative paths to the files with demonstration data to load. These will only be loaded if the database was created with the Demo Data flag enabled. The image that is used as the module icon is the PNG file at static/description/icon.png. Odoo is expected to have significant changes between major versions, so modules built for one major version are likely to not be compatible with the next version without conversion and migration work. Due to this, it's important to be sure about a module's Odoo target version before installing it. There's more… Instead of having the long description in the module manifest, it's possible to have it in its own file. Since version 8.0, it can be replaced by a README file, with either a .txt, .rst, or an .md (Markdown) extension. Otherwise, include a description/index.html file in the module. This HTML description will override a description defined in the manifest file. There are a few more keys that are frequently used: application: If this is True, the module is listed as an application. Usually, this is used for the central module of a functional area auto_install: If this is True, it indicates that this is a "glue" module, which is automatically installed when all of its dependencies are installed installable: If this is True (the default value), it indicates that the module is available for installation Organizing the addon module file structure An addon module contains code files and other assets such as XML files and images. For most of these files, we are free to choose where to place them inside the module directory. However, Odoo uses some conventions on the module structure, so it is advisable to follow them. Getting ready We are expected to have an addon module directory with only the __init__.py and __manifest__.py files. In this recipe, we suppose this is local-addons/my_module. How to do it... To create the basic skeleton for the addon module, perform the given steps: Create the directories for code files: $ cd local-addons/my_module $ mkdir models $ touch models/__init__.py $ mkdir controllers $ touch controllers/__init__.py $ mkdir views $ mkdir security $ mkdir data $ mkdir demo $ mkdir i18n $ mkdir -p static/description Edit the module's top __init__.py file so that the code in subdirectories is loaded: from . import models from . import controllers This should get us started with a structure containing the most used directories, similar to this one: . ├── __init__.py ├── __manifest__.py │ ├── controllers │ └── __init__.py ├── data ├── demo ├── i18n ├── models │ └── __init__.py ├── security ├── static │ └── description └── views How it works... To provide some context, an Odoo addon module can have three types of files:  The Python code is loaded by the __init__.py files, where the .py files and code subdirectories are imported. Subdirectories containing code Python, in turn, need their own __init__.py. Data files that are to be declared in the data and demo keys of the __manifest__.py module manifest in order to be loaded are usually XML and CSV files for the user interface, fixture data, and demonstration data. There can also be YAML files, which can include some procedural instructions that are run when the module is loaded, for instance, to generate or update records programmatically rather than statically in an XML file. Web assets such as JavaScript code and libraries, CSS, and QWeb/HTML templates also play an important part. There are declared through an XML file extending the master templates to add these assets to the web client or website pages. The addon files are to be organized in these directories: models/ contains the backend code files, creating the Models and their business logic. A file per Model is recommended, with the same name as the model, for example, library_book.py for the library.book model. views/ contains the XML files for the user interface, with the actions, forms, lists, and so on. As with models, it is advised to have one file per model. Filenames for website templates are expected to end with the _template suffix. data/ contains other data files with module initial data. demo/ contains data files with demonstration data, useful for tests, training, or module evaluation. i18n/ is where Odoo will look for the translation .pot and .po files. These files don't need to be mentioned in the manifest file. security/ contains the data files defining access control lists, usually a ir.model.access.csv file, and possibly an XML file to define access Groups and Record Rules for row level security. controllers/ contains the code files for the website controllers, and for modules providing that kind of feature. static/ is where all web assets are expected to be placed. Unlike other directories, this directory name is not just a convention, and only files inside it can be made available for the Odoo web pages. They don't need to be mentioned in the module manifest, but will have to be referred to in the web template. When adding new files to a module, don't forget to declare them either in the __manifest__.py (for data files) or __init__.py (for code files); otherwise, those files will be ignored and won't be loaded. Adding models Models define the data structures to be used by our business applications. This recipe shows how to add a basic model to a module. We will use a simple book library example to explain this; we want a model to represent books. Each book has a name and a list of authors. Getting ready We should have a module to work with. We will use an empty my_module for our explanation. How to do it... To add a new Model, we add a Python file describing it and then upgrade the addon module (or install it, if it was not already done). The paths used are relative to our addon module location (for example, ~/odoo-dev/local-addons/my_module/): Add a Python file to the models/library_book.py module with the following code: from odoo import models, fields class LibraryBook(models.Model): _name = 'library.book' name = fields.Char('Title', required=True) date_release = fields.Date('Release Date') author_ids = fields.Many2many( 'res.partner', string='Authors' ) Add a Python initialization file with code files to be loaded by the models/__init__.py module with the following code: from . import library_book Edit the module Python initialization file to have the models/ directory loaded by the module: from . import models Upgrade the Odoo module, either from the command line or from the apps menu in the user interface. If you look closely at the server log while upgrading the module, you should see this line: odoo.modules.registry: module my_module: creating or updating database table After this, the new library.book model should be available in our Odoo instance. If we have the technical tools activated, we can confirm that by looking it up at Settings | Technical | Database Structure | Models. How it works... Our first step was to create a Python file where our new module was created. Odoo models are objects derived from the Odoo Model Python class. When a new model is defined, it is also added to a central model registry. This makes it easier for other modules to make modifications to it later. Models have a few generic attributes prefixed with an underscore. The most important one is _name, providing a unique internal identifier to be used throughout the Odoo instance. The model fields are defined as class attributes. We began defining the name field of the Char type. It is convenient for models to have this field, because by default, it is used as the record description when referenced from other models. We also used an example of a relational field, author_ids. It defines a many-to-many relation between Library Books and the partners. A book can have many authors and each author can have written many books. Next, we must make our module aware of this new Python file. This is done by the __init__.py files. Since we placed the code inside the models/ subdirectory, we need the previous __init__ file to import that directory, which should, in turn, contain another __init__ file importing each of the code files there (just one, in our case). Changes to Odoo models are activated by upgrading the module. The Odoo server will handle the translation of the model class into database structure changes. Although no example is provided here, business logic can also be added to these Python files, either by adding new methods to the Model's class or by extending the existing methods, such as create() or write(). Thus we learned about the structure of an Odoo addon module and learned, step-by-step, how to create a simple module from scratch. To know more about, how to define access rules for your data;  how to expose your data models to end users on the back end and on the front end; and how to create beautiful PDF versions of your data, read this book Odoo 11 Development Cookbook - Second Edition. ERP tool in focus: Odoo 11 Building Your First Odoo Application How to Scaffold a New module in Odoo 11

In this article by Navin Agarwal, author of the book Liferay Portal 6.2 Enterprise Intranets, we will learn that Liferay is an enterprise application solution. It provides a lot of functionalities, which helps an organization to grow and is a one-solution package as a portal and content management solution. In this article, we will look at the following topics: The complete features you want your organization's intranet solution to have Reasons why Liferay is an excellent choice to build your intranet Where and how Liferay is used besides intranet portals Easy integration with other open source tools and applications Getting into more technical information about what Liferay is and how it works So, let's start looking at exactly what kind of site we're going to build. (For more resources related to this topic, see here.) Liferay Portal makes life easy We're going to build a complete corporate intranet solution using Liferay. Let's discuss some of the features your intranet portal will have. Hosted discussions Are you still using e-mail for group discussions? Then, it's time you found a better way! Running group discussions over e-mail clogs up the team's inbox—this means you have to choose your distribution list in advance, and that makes it hard for team members to opt in and out of the discussion. Using Liferay, we will build a range of discussion boards for discussion within and between teams. The discussions are archived in one place, which means that it's always possible to go back and refer to them later. On one level, it's just more convenient to move e-mail discussions to a discussion forum designed for the purpose. But once the forum is in place, you will find that a more productive group discussion takes place here than it ever did over e-mail. Collaborative documents using wikis Your company probably has guideline documents that should be updated regularly but swiftly lose their relevance as practices and procedures change. Even worse, each of your staff will know useful, productive tricks and techniques—but there's probably no easy way to record that knowledge in a way that is easy for others to find and use. We will see how to host wikis within Liferay. A wiki enables anybody to create and edit web pages and link all of those web pages together without requiring any HTML or programming skills. You can put your guideline documents into a wiki, and as practices change, your frontline staff can quickly and effortlessly update the guideline documentation. Wikis can also act as a shared notebook, enabling team members to collaborate and share ideas and findings and work together on documents. Team and individual blogs Your company probably needs frequent, chronological publications of personal thoughts and web links in the intranet. Your company probably has teams and individuals working on specific projects in order to share files and blogs about a project process and more. By using the Liferay Blog features, you can use HTML text editors to create or update files and blogs and to provide RSS feeds. Liferay provides an easy way for teams and individuals to share files with the help of blogs. Blogs provide a straightforward blogging solution with features such as RSS, user and guest comments, browsable categories, tags and labels, and a rating system. Liferay's RSS with the subscription feature provides the ability to frequently read RSS feeds from within the portal framework. At the same time, What You See Is What You Get (WYSIWYG) editors provide the ability to edit web content, including the blogs' content. Less technical people can use the WYSIWYG editor instead of sifting through complex code. Shared calendars Many companies require calendar information and share the calendar among users from different departments. We will see how to share a calendar within Liferay. The shared calendar can satisfy the basic business requirements incorporated into a featured business intranet, such as scheduling meetings, sending meeting invitations, checking for attendees' availability, and so on. Therefore, you can provide an environment for users to manage events and share calendars. Document management – CMS When there is a need for document sharing and document management, Liferay's Documents and Media library helps you with lots of features. The Documents and Media portlet allows you to add folders and subfolders for documents and media files, and also allows users to publish documents. It serves as a repository for all types of files and makes Content management systems (CMSes) available for intranets. The Documents and Media library portlet is equipped with customizable folders and acts as a web-based solution to share documents and media files among all your team members—just as a shared drive would. All the intranet users will be able to access the files from anywhere, and the content is accessible only by those authorized by administrators. All the files are secured by the permission layer by the administrator. Web content management – WCM Your company may have a lot of images and documents, and you may need to manage all these images and documents as well. Therefore, you require the ability to manage a lot of web content and then publish web content in intranets. We will see how to manage web content and how to publish web content within Liferay. Liferay Journal (Web Content) not only provides high availability to publish, manage, and maintain web content and documents, but it also separates content from the layout. Liferay WCM allows us to create, edit, and publish web content (articles). It also allows quick changes in the preview of the web content by changing the layout. It has built-in functionality, such as workflow, search, article versioning, scheduling, and metadata. Personalization and internalization All users can get a personal space that can be either made public (published as a website with a unique, friendly URL) or kept private. You can also customize how the space looks, what tools and applications are included, what goes into Documents and Media, and who can view and access all of this content. In addition, Liferay supports multiple languages, where you can select your own language. Multilingual organizations get out-of-the-box support for up to 45 languages. Users can toggle among different language settings with just one click and produce/publish multilingual documents and web content. Users can make use of the internalization feature to define the specific site in a localized language. Workflow, staging, scheduling, and publishing You can use a workflow to manage definitions, instances, and predetermined sequences of connected steps. Workflow can be used for web content management, assets, and so on. Liferay's built-in workflow engine is called Kaleo. It allows users to set up the review and publishing process on the web content article of any document that needs to end up on the live site. Liferay 6.2 integrates with the powerful features of the workflow and data capabilities of dynamic data lists in Kaleo Forms; it's only available in Liferay Enterprise Edition. Staging environments are integrated with Liferay's workflow engine. To have a review process for staged pages, you need to make sure you have a workflow engine configured and you have a staging setup in the workflow. As a content creator, you can update what you've created and publish it in a staging workflow. Other users can then review and modify it. Moreover, content editors can make a decision on whether to publish web content from staging to live, that is, you can easily create and manage everything from a simple article of text and images to fully functional websites in staging and then publish them live. Before going live, you can schedule web content as well. For instance, you can publish web content immediately or schedule it for publishing on a specific date. Social networks and Social Office Liferay Portal supports social networks—you can easily manage your Google Plus, Facebook, MySpace, Twitter, and other social network accounts in Liferay. In addition, you can manage your instant messenger accounts, such as AIM, ICQ, Jabber, MSN, Skype, YM, and so on smoothly from inside Liferay. Liferay Social Office gives us a social collaboration on top of the portal—a fully virtual workspace that streamlines communication and builds up group cohesion. It provides holistic enhancement to the way you and your colleagues work together. All components in Social Office are tied together seamlessly, getting everyone on the same page by sharing the same look and feel. More importantly, the dynamic activity tracking gives us a bird's-eye view of who has been doing what and when within each individual site. Using Liferay Social Office, you can enhance your existing personal workflow with social tools, keep your team up to date, and turn collective knowledge into collective action. Note that Liferay 6.2 supports the Liferay Social Office 3.0 current version. Liferay Sync and Marketplace Liferay Sync is Liferay's newest product, designed to make file sharing as easy as a simple drag and drop! Liferay Sync is an add-on product for Liferay 6.1 CE, EE, and later versions, which makes it a more raw boost product and enables the end user to publish and access documents and files from multiple environments and devices, including Windows and MacOS systems, and iOS-based mobile platforms. Liferay Sync is one of the best features, and it is fully integrated into the Liferay platform. Liferay 6.1 introduced the new concept of the marketplace, which leverages the developers to develop any components or functionality and release and share it with other users. It's a user-friendly and one-stop place to share apps. Liferay Marketplace provides the portal product with add-on features with a new hub to share, browse, and download Liferay-compatible applications. In Liferay 6.2, Marketplace comes under App Manager, where all the app-related controls can be possible. More features The intranet also arranges staff members into teams and sites, provides a way of real-time IM and chatting, and gives each user an appropriate level of access. This means that they can get all the information they need and edit and add content as necessary but won't be able to mess with sensitive information that they have no reason to see. In particular, the portal provides an integrating framework so that you can integrate external applications easily. For example, you can integrate external applications with the portal, such as Alfresco, OpenX, LDAP, SSO CAS, Orbeon Forms, Konakart, PayPal, Solr, and so on. In a word, the portal offers compelling benefits to today's enterprises—reduced operational costs, improved customer satisfaction, and streamlined business processes. Everything in one place All of these features are useful on their own. However, it gets better when you consider that all of these features will be combined into one easy-to-use searchable portal. A user of the intranet, for example, can search for a topic—let's say financial report—and find the following in one go: Any group discussions about financial reports Blog entries within the intranet concerning financial reports Documents and files—perhaps the financial reports themselves Wiki entries with guidelines on preparing financial reports Calendar entries for meetings to discuss the financial report Of course, users can also restrict their search to just one area if they already know exactly what they are looking for. Liferay provides other features, such as tagging, in order to make it even easier to organize information across the whole intranet. We will do all of this and more. Introducing Palm Tree Publications We are going to build an intranet for a fictional company as an example, focusing on how to install, configure, and integrate it with other applications and also implement portals and plugins (portlets, themes, layout templates, hooks, and webs) within Liferay. By applying the instructions to your own business, you will be able to build an intranet to meet your own company's needs. "Palm Tree Publications" needs an intranet of its own, which we will call bookpub.com. The enterprise's global headquarters are in the United States. It has several departments—editorial, website, engineering, marketing, executive, and human resources. Each department has staff in the U.S., Germany, and India or in all three places. The intranet site provides a site called "Book Street and Book Workshop" consisting of users who have an interest in reading books. The enterprise needs to integrate collaboration tools, such as wikis, discussion forums, blogs, instant messaging, mail, RSS, shared calendars, tagging, and so on. Palm Tree Publications has more advanced needs too: a workflow to edit, approve, and publish books. Furthermore, the enterprise has a lot of content, such as books stored and managed alfresco currently. In order to build the intranet site, the following functionality should be considered: Installing the portal, experiencing the portal and portlets, and customizing the portal and personal web pages Bringing the features of enabling document sharing, calendar sharing, and other collaboration within a business to the users of the portal Discussion forums—employees should be able to discuss book ideas and proposals Wikis—keeping track of information about editorial guidance and other resources that require frequent editing Dissemination of information via blogs—small teams working on specific projects share files and blogs about a project process Sharing a calendar among employees Web content management creation by the content author and getting approved by the publisher Document repository—using effective content management systems (CMSes), a natural fit for a portal for secure access, permissions, and distinct roles (such as writers, editors, designers, administrators, and so on) Collaborative chat and instant messaging, social network, Social Office, and knowledge management tools Managing a site named Book Street and Book Workshop that consists of users who have the same interest in reading books as staging, scheduling, and publishing web content related to books Federated search for discussion forum entries, blog posts, wiki articles, users in the directory, and content in both the Document and Media libraries; search by tags Integrating back-of-the-house software applications, such as Alfresco, Orbeon Forms, the Drools rule server, Jasper Server, and BI/Reporting Pentaho; strong authentication and authorization with LDAP; and single authentication to access various company sites besides the intranet site The enterprise can have the following groups of people: Admin: This group installs systems, manages membership, users, user groups, organizations, roles and permissions, security on resources, workflow, servers and instances, and integrates with third-party systems Executives: Executive management handles approvals Marketing: This group handles websites, company brochures, marketing campaigns, projects, and digital assets Sales: This group makes presentations, contracts, documents, and reports Website editors: This group manages pages of the intranet—writes articles, reviews articles, designs the layout of articles, and publishes articles Book editors: This group writes, reviews, and publishes books and approves and rejects the publishing of books Human resources: This group manages corporate policy documents Finance: This group manages accounts documents, scanned invoices and checks accounts Corporate communications: This group manages external public relations, internal news releases, and syndication Engineering: This group sets up the development environment and collaborates on engineering projects and presentation templates Introducing Liferay Portal's architecture and framework Liferay Portal's architecture supports high availability for mission-critical applications using clustering and the fully distributed cache and replication support across multiple servers. The following diagram has been taken from the Liferay forum written by Jorge Ferrer. This diagram depicts the various architectural layers and functionalities of portlets: Figure 1.1: The Liferay architecture The preceding image was taken from https://www.liferay.com/web/jorge.ferrer/blog/-/blogs/liferay-s-architecture-the-beginning-of-a-blog-series site blog. The Liferay Portal architecture is designed in such a way that it provides tons of features at one place: Frontend layer: This layer is the end user's interface Service layer: This contains the great majority of the business logic for the portal platform and all of the portlets included out of the box Persistence layer: Liferay relies on Hibernate to do most of its database access Web services API layer: This handles web services, such as JSON and SOAP In Liferay, the service layer, persistence layer, and web services API layer are built automatically by that wonderful tool called Service Builder. Service Builder is the tool that glues together all of Liferay's layers and that hides the complexities of using Spring or Hibernate under the hood. Service-oriented architecture Liferay Portal uses service-oriented architecture (SOA) design principles throughout and provides the tools and framework to extend SOA to other enterprise applications. Under the Liferay enterprise architecture, not only can the users access the portal from traditional and wireless devices, but developers can also access it from the exposed APIs via REST, SOAP, RMI, XML-RPC, XML, JSON, Hessian, and Burlap. Liferay Portal is designed to deploy portlets that adhere to the portlet API compliant with both JSR-168 and JSR-286. A set of useful portlets are bundled with the portal, including Documents and Media, Calendar, Message Boards, Blogs, Wikis, and so on. They can be used as examples to add custom portlets. In a word, the key features of Liferay include using SOA design principles throughout, such as reliable security, integrating the portal with SSO and LDAP, multitier and limitless clustering, high availability, caching pages, dynamic virtual hosting, and so on. Understanding Enterprise Service Bus Enterprise Service Bus (ESB) is a central connection manager that allows applications and services to be added quickly to an enterprise infrastructure. When an application needs to be replaced, it can easily be disconnected from the bus at a single point. Liferay Portal uses Mule or ServiceMix as ESB. Through ESB, the portal can integrate with SharePoint, BPM (such as the jBPM workflow engine and Intalio | BPMS engine), BI Xforms reporting, JCR repository, and so on. It supports JSR 170 for content management systems with the integration of JCR repositories, such as Jackrabbit. It also uses Hibernate and JDBC to connect to any database. Furthermore, it supports an event system with synchronous and asynchronous messaging and a lightweight message bus. Liferay Portal uses the Spring framework for its business and data services layers. It also uses the Spring framework for its transaction management. Based on service interfaces, portal-impl is implemented and exposed only for internal usage—for example, they are used for the extension environment. portal-kernel and portal-service are provided for external usage (or for internal usage)—for example, they are used for the Plugins SDK environment. Custom portlets, both JSR-168 and JSR-286, and web services can be built based on portal-kernel and portal-service. In addition, the Web 2.0 Mail portlet and the Web 2.0 Chat portlet are supported as well. More interestingly, scheduled staging and remote staging and publishing serve as a foundation through the tunnel web for web content management and publishing. Liferay Portal supports web services to make it easy for different applications in an enterprise to communicate with each other. Java, .NET, and proprietary applications can work together easily because web services use XML standards. It also supports REST-style JSON web services for lightweight, maintainable code and supports AJAX-based user interfaces. Liferay Portal uses industry-standard, government-grade encryption technologies, including advanced algorithms, such as DES, MD5, and RSA. Liferay was benchmarked as one of the most secure portal platforms using LogicLibrary's Logiscan suite. Liferay offers customizable single sign-on (SSO) that integrates into Yale CAS, JAAS, LDAP, NTLM, CA Siteminder, Novell Identity Manager, OpenSSO, and more. Open ID, OpenAuth, Yale CAS, Siteminder, and OpenAM integration are offered by it out of the box. In short, Liferay Portal uses ESB in general with an abstraction layer on top of an enterprise messaging system. It allows integration architects to exploit the value of messaging systems, such as reporting, e-commerce, and advertisements. Understanding the advantages of using Liferay to build an intranet Of course, there are lots of ways to build a company intranet. What makes Liferay such a good choice to create an intranet portal? It has got the features we need All of the features we outlined for our intranet come built into Liferay: discussions, wikis, calendars, blogs, and so on are part of what Liferay is designed to do. It is also designed to tie all of these features together into one searchable portal, so we won't be dealing with lots of separate components when we build and use our intranet. Every part will work together with others. Easy to set up and use Liferay has an intuitive interface that uses icons, clear labels, and drag and drop to make it easy to configure and use the intranet. Setting up the intranet will require a bit more work than using it, of course. However, you will be pleasantly surprised by how simple it is—no programming is required to get your intranet up and running. Free and open source How much does Liferay cost? Nothing! It's a free, open source tool. Here, being free means that you can go to Liferay's website and download it without paying anything. You can then go ahead and install it and use it. Liferay comes with an enterprise edition too, for which users need to pay. In addition, Liferay provides full support and access to additional enterprise edition plugins/applications. Liferay makes its money by providing additional services, including training. However, the standard use of Liferay is completely free. Now you probably won't have to pay another penny to get your intranet working. Being open source means that the program code that makes Liferay work is available to anybody to look at and change. Even if you're not a programmer, this is still good for you: If you need Liferay to do something new, then you can hire a programmer to modify Liferay to do it. There are lots of developers studying the source code, looking for ways to make it better. Lots of improvements get incorporated into Liferay's main code. Developers are always working to create plugins—programs that work together with Liferay to add new features. Probably, for now, the big deal here is that it doesn't cost any money. However, as you use Liferay more, you will come to understand the other benefits of open source software for you. Grows with you Liferay is designed in a way that means it can work with thousands and thousands of users at once. No matter how big your business is or how much it grows, Liferay will still work and handle all of the information you throw at it. It also has features especially suited to large, international businesses. Are you opening offices in non-English speaking countries? No problem! Liferay has internationalization features tailored to many of the world's popular languages. Works with other tools Liferay is designed to work with other software tools—the ones that you're already using and the ones that you might use in the future—for instance: You can hook up Liferay to your LDAP directory server and SSO so that user details and login credentials are added to Liferay automatically Liferay can work with Alfresco—a popular and powerful Enterprise CMS (used to provide extremely advanced document management capabilities, which are far beyond what Liferay does on its own) Based on "standards" This is a more technical benefit; however, it is a very useful one if you ever want to use Liferay in a more specialized way. Liferay is based on standard technologies that are popular with developers and other IT experts and that confer the following benefits on users: Built using Java: Java is a popular programming language that can run on just about any computer. There are millions of Java programmers in the world, so it won't be too hard to find developers who can customize Liferay. Based on tried and tested components: With any tool, there's a danger of bugs. Liferay uses lots of well-known, widely tested components to minimize the likelihood of bugs creeping in. If you are interested, here are some of the well-known components and technologies Liferay uses—Apache ServiceMix, Mule, ehcache, Hibernate, ICEfaces, Java J2EE/JEE, jBPM, Activiti, JGroups, Alloy UI, Lucene, PHP, Ruby, Seam, Spring and AOP, Struts and Tiles, Tapestry, Velocity, and FreeMarker. Uses standard ways to communicate with other software: There are various standards established to share data between pieces of software. Liferay uses these so that you can easily get information from Liferay into other systems. The standards implemented by Liferay include AJAX, iCalendar and Microformat, JSR-168, JSR-127, JSR-170, JSR-286 (Portlet 2.0), JSR-314 (JSF 2.0), OpenSearch, the Open platform with support for web services, including JSON, Hessian, Burlap, REST, RMI, and WSRP, WebDAV, and CalDAV. Makes publication and collaboration tools Web Content Accessibility Guidelines 2.0 (WCAG 2.0) compliant: The new W3C recommendation is to make web content accessible to a wide range of people with disabilities, including blindness and low vision, deafness and hearing loss, learning disabilities, cognitive limitations, limited movement, speech disabilities, photosensitivity, and combinations of these. For example, the portal integrates CKEditor-standards support, such as W3C (WAI-AA and WCAG), 508 (Section 508). Alloy UI: The Liferay UI supports HTML 5, CSS 3, and Yahoo! User Interface Library 3 (YUI 3). Supports Apache Ant 1.8 and Maven 2: Liferay Portal can be built through Apache Ant by default, where you can build services; clean, compile, and build JavaScript CMD; build language native to ASCII, deploy, fast deploy; and so on. Moreover, Liferay supports Maven 2 SDK, providing Community Edition (CE) releases through public maven repositories as well as Enterprise Edition (EE) customers to install maven artifacts in their local maven repository. Bootstrap: Liferay 6.2 provides support for Twitter Bootstrap out of the box. With its fully responsive UI, the benefit of bootstrap is that it will support any device to render the content. Even content authors can use bootstrap markup and styles to make the content nicer. Many of these standards are things that you will never need to know much about, so don't worry if you've never heard of them. Liferay is better for using them, but mostly, you won't even know they are there. Other advantages of Liferay Liferay isn't just for intranets! Users and developers are building all kinds of different websites and systems based on Liferay. Corporate extranets An intranet is great for collaboration and information sharing within a company. An extranet extends this facility to suppliers and customers, who usually log in over the Internet. In many ways, this is similar to an intranet—however, there are a few technical differences. The main difference is that you create user accounts for people who are not part of your company. Collaborative websites Collaborative websites not only provide a secure and administrated framework, but they also empower users with collaborative tools, such as blogs, instant e-mail, message boards, instant messaging, shared calendars, and so on. Moreover, they encourage users to use other tools, such as tag administration, fine-grained permissions, delegable administrator privileges, enterprise taxonomy, and ad hoc user groups. By means of these tools, as an administrator, you can ultimately control what people can and cannot do in Liferay. In many ways, this is similar to an intranet too; however, there are a few technical differences. The main difference is that you use collaborative tools simply, such as blogs, instant e-mail, message boards, instant messaging, shared calendars, and so on. Content management and web publishing You can also use Liferay to run your public company website with content management and web publishing. Content management and web publishing are useful features in websites. It is a fact that the volume of digital content for any organization is increasing on a daily basis. Therefore, an effective CMS is a vital part of any organization. Meanwhile, document management is also useful and more effective when repositories have to be assigned to different departments and groups within the organization. Content management and document management are effective in Liferay. Moreover, when managing and publishing content, we may have to answer many questions, such as "who should be able to update and delete a document from the system?". Fortunately, Liferay's security and permissions model can satisfy the need for secure access and permissions and distinct roles (for example, writer, editor, designer, and administrator). Furthermore, Liferay integrates with the workflow engine. Thus, users can follow a flow to edit, approve, and publish content in the website. Content management and web publishing are similar to an intranet; however, there are a few technical differences. The main difference is that you can manage content and publish web content smoothly. Infrastructure portals Infrastructure portals integrate all possible functions, as we stated previously. This covers collaboration and information sharing within a company in the form of collaborative tools, content management, and web publishing. In infrastructure portals, users can create a unified interface to work with content, regardless of source via content interaction APIs. Furthermore, using the same API and the same interface as that of the built-in CMS, users can also manage content and publish web content from third-party systems, such as Alfresco, Vignette, Magnolia, FatWire, Microsoft SharePoint, and so on. Infrastructure portals are similar to an intranet; there are a few technical differences though. The main difference is that you can use collaborative tools, manage content, publish web content, and integrate other systems in one place. Why do you need a portal? The main reason is that a portal can serve as a framework to aggregate content and applications. A portal normally provides a secure and manageable framework where users can easily make new and existing enterprise applications available. In order to build an infrastructure portal smoothly, Liferay Portal provides an SOA-based framework to integrate third-party systems. Out-of-the-box portlets and features Liferay provides out-of-the-box (OOTB) portlets that have key features and can be used in the enterprise intranet very efficiently. These portlets are very scalable and powerful and provide the developer with the tools to customize it very easily. Let's see some of the most frequently used portlets in Liferay Portal. Content management Content management is a common feature in any web-based portal or website: The Web Content portlet has the features of full web publishing, office integration, and the asset library, which contains documents, images, and videos. This portlet also has the structure and templates that help with the designing of the web content's look and feel. Structure can be designed with the help of a visual editor with drag and drop. It has the integrated help feature with tooltips to name the attributes of the fields. The Asset Publisher portlet provides you with the feature to select any type of content/asset, such as wiki pages, web content, calendar events, message board messages, documents, media documents, and many more. It also allows us to use filter on them by types, categories, tags, and sources. The display settings provide configurable settings, which helps the content to be displayed to the end users perfectly. The Document and Media portlet is one of the most usable portlets to store any type of document. It allows you to store and manage your documents. It allows you to manage Liferay documents from your own machine's filesystem with the help of WebDAV integration. It has lots of new, built-in features, such as the inline document preview, image preview, and video player. Document metadata is displayed in document details, which makes it easier for you to review the metadata of the document. Also, Document and Media has features named checkin and checkout that helps editing the document in a group very easily. The Document and Media portlet has the multi-repository integration feature, which allows you to configure or mount any other repository very easily, such as SharePoint, Documentum, and Alfresco, utilizing the CMIS standard. Collaboration Collaboration features are generally ways in which users communicate with each other, such as the ones shown in the following list: The Dynamic data list portlet provides you with the facility of not writing a single line of code to create the form or data list. Say, for example, your corporate intranet needs the job posting done on a daily basis by the HR administrator. The administrator needs to develop the custom portlet to fulfill that requirement. Now, the dynamic data list portlet will allow the administrator to create a form for job posting. It's very easy to create and display new data types. The Blog portlet is one of the best features of Liferay. Blog portlets have two other related portlets, namely Recent Bloggers and Blogs Aggregator. The blog portlet provides the best possible ways for chronological publications of personal thoughts and web links in the intranet. Blog portlets can be placed for users of different sites/departments under the respective site//department page. The Calendar portlet provides the feature to create the event and schedule the event. It has many features that help the users in viewing the meeting schedule. The Message Board portlet is a full-featured forum solution with threaded views, categories, RSS capability, avatars, file attachments, previews, dynamic lists of recent posts, and forum statistics. Message Board portlets work with the fine-grained permissions and role-based access control model to give detailed levels of control to administrators and users. The Wiki portlet, like the Message Boards portlet, provides a straightforward wiki solution for both intranet and extranet portals that provides knowledge management among the users. It has all of the features you would expect in a state-of-the-art wiki. Again, it has the features of a file attachment preview, publishing the content, and versioning, and works with a fine-grained permission and role-based access control model. This again takes all the features of the Liferay platform. The Social Activity portlet allows you to tweak the measurements used to calculate user involvement within a site. The contribution and participation values determine the reward value of an action. It uses the blog entry, wiki, and message board points to calculate the user involvement in the site. The Marketplace portlet is placed inside the control panel. It's a hub for the applications provided by Liferay and other partners. You can find that many applications are free, and for certain applications, you need to pay an amount. It's more like an app store. This feature was introduced in Liferay Version 6.1. In the Liferay 6.2 control panel, under the Apps | Store link section, you will see apps that are stored in the Marketplace portlet. Liferay 6.2 comes with a new control panel that is very easy to manage for the portal's Admin users. Liferay Sync is not a portlet; it's a new feature of Liferay that allows you to synchronize documents of Liferay Document and Media with your local system. Liferay provide the Liferay Sync application, which has to be installed in your local system or mobile device. News RSS portlets provide RSS feeds. RSS portlets are used for the publishers by letting them syndicate content automatically. They benefit readers who want to subscribe to timely updates from their favorite websites or to aggregate feeds from many sites into one place. A Liferay RSS portlet is fully customizable, and it allows you to set the URL from which site you would like to get feeds. Social Activities portlets display portal-wide user activity, such as posting on message boards, creating wikis, and adding documents to Documents and Media. There are more portlets for social categories, such as User Statistics portlets, Group Statistics portlets, and Requests portlets. All these portlets are used for the social media. Tools The Search portlet provides faceted search features. When a search is performed, facet information will appear based on the results of the search. The number of each asset type and the most frequently occurring tags and categories as well as their frequency will all appear in the left-hand side column of the portlet. It searches through Bookmarks, Blogs Entries, Web Content Articles, Document Library Files, Users, Message Board, and Wiki. Finding more information on Liferay In this article, we looked at what Liferay can do for your corporate intranet and briefly saw why it's a good choice. If you want more background information on Liferay, the best place to start is the Liferay corporate website (http://www.liferay.com) itself. You can find the latest news and events, various training programs offered worldwide, presentations, demonstrations, and hosted trails. More interestingly, Liferay eats its own dog food; corporate websites within forums (called message boards), blogs, and wikis are built by Liferay using its own products. It is a real demo of Liferay Portal's software. Liferay is 100 percent open source and all downloads are available from the Liferay Portal website at http://www.liferay.com/web/guest/downloads/portal and the SourceForge website at http://sourceforge.net/projects/lportal/files. The source code repository is available at https://github.com/liferay. The Liferay website's wiki (http://www.liferay.com/web/guest/community/wiki) contains documentation, including a tutorial, user guide, developer guide, administrator guide, roadmap, and so on. The Liferay website's discussion forums can be accessed at http://www.liferay.com/web/guest/community/forums and the blogs at http://www.liferay.com/community/blogs/highlighted. The official plugins and the community plugins are available at http://www.liferay.com/marketplace and are the best place to share your thoughts, get tips and tricks about Liferay implementation, and use and contribute community plugins. If you would like to file a bug or know more about the fixes in a specific release, then you must visit the bug-tracking system at http://issues.liferay.com/. Summary In this article, we looked at what Liferay can offer your intranet and what we should consider while designing the company's enterprise site. We saw that our final intranet will provide shared documents, discussions, collaborative wikis, and more in a single, searchable portal. Well, Liferay is a great choice for an intranet because it provides so many features and is easy to use, free and open source, extensible, and well-integrated with other tools and standards. We also saw the other kinds of sites Liferay is good for, such as extranets, collaborative websites, content management, web publishing, and infrastructure portals. For the best example of an intranet and extranet, you can visit www.liferay.com. It will provide you with more background information. Resources for Article: Further resources on this subject: Working with a Liferay User / User Group / Organization[article] Liferay, its Installation and setup[article] Building your First Liferay Site [article]

Learning new skills takes time - that's why, before learning something, you need to know that what you're learning is going to be worthwhile. This is particularly true when deciding which programming language to learn. As we approach the new year, it's a good time to reflect on our top learning priorities for 2019. But which programming should you learn in 2019? We’ve put together a list of the top programming languages to learn in the new year - as well as reasons you should learn them, and some suggestions for how you can get started. This will help you take steps to expand your skill set in 2019 in the way that’s right for you. Want to learn a new programming language? We have thousands of eBooks and videos in our $5 sale to help you get to grips with everything from Python to Rust. Python Python has been a growing programming language for some time and it shows no signs of disappearing. There are a number of reasons for this, but the biggest is the irresistible allure of artificial intelligence. Once you know Python, performing some relatively sophisticated deep learning tasks becomes relatively easy, not least because of the impressive ecosystem of tools that surround it, like TensorFlow. But Python’s importance isn’t just about machine learning. It’s flexibility means it has a diverse range of applications. If you’re a full-stack developer, for example, you might find Python useful for developing backend services and APIs; equally, if you’re in security or SRE, Python can be useful for automating aspects of your infrastructure to keep things secure and reliable. Put simply, Python is a useful addition to your skill set. Learn Python in 2019 if... You’re new to software development You want to try your hand at machine learning You want to write automation scripts Want to get started? Check out these titles: Clean Code in Python Learning Python Learn Programming in Python with Cody Jackson Python for Everyday Life [Video]       Go Go isn’t quite as popular as Python, but it is growing rapidly. And its fans are incredibly vocal about why they love it: it’s incredibly simple, yet also amazingly powerful. The reason for this is its creation: it was initially developed by Google that wanted a programming language that could handle the complexity of the systems they were developing, without adding to complexity in terms of knowledge and workflows. Combining the best aspects of functional and object oriented programming, as well as featuring a valuable set of in-built development tools, the language is likely to only go from strength to strength over the next 12 months. Learn Go in 2019 if… You’re a backend or full-stack developer looking to increase your language knowledge You’re working in ops or SRE Looking for an alternative to Python Learn Go with these eBooks and videos: Mastering Go Cloud Native programming with Golang Hands-On Go Programming Hands-On Full-Stack Development with Go       Rust In Stack Overflow’s 2018 developer survey Rust was revealed to be the best loved language among the developers using it. 80% of respondents said they loved using it or wanted to use it. Now, while Rust lacks the simplicity of Go and Python, it does do what it sets out to do very well - systems programming that’s fast, efficient, and secure. In fact, developers like to debate the merits of Rust and Go - it seems they occupy the minds of very similar developers. However, while they do have some similarities, there are key differences that should make it easier to decide which one you learn. At a basic level, Rust is better for lower level programming, while Go will allow you to get things done quickly. Rust does have a lot of rules, all of which will help you develop incredibly performant applications, but this does mean it has a steeper learning curve than something like Go. Ultimately it will depend on what you want to use the language for and how much time you have to learn something new. Learn Rust in 2019 if… You want to know why Rust developers love it so much You do systems programming You have a bit of time to tackle its learning curve Learn Rust with these titles: Rust Quick Start Guide Building Reusable Code with Rust [Video] Learning Rust [Video] Hands-On Concurrency with Rust       TypeScript TypeScript has quietly been gaining popularity over recent years. But it feels like 2018 has been the year that it has really broke through to capture the imagination of the wider developer community. Perhaps it’s just Satya Nadella’s magic... More likely, however, it’s because we’re now trying to do too much with plain old JavaScript. We simply can’t build applications of the complexity we want without drowning in lines of code. Essentially, TypeScript bulks up JavaScript, and makes it suitable for building applications of the future. It’s no surprise that TypeScript is now fundamental to core JavaScript frameworks - even Google decided to use it in Angular. But it’s not just for front end JavaScript developers - there are examples of Java and C# developers looking closely at TypeScript, as it shares many features with established statically typed languages. Learn TypeScript in 2019 if… You’re a JavaScript developer You’re a Java or C# developer looking to expand their horizons Learn TypeScript in 2019: TypeScript 3.0 Quick Start Guide TypeScript High Performance Introduction to TypeScript [Video]         Scala Scala has been around for some time, but its performance gains over Java have seen it growing in popularity in recent years. It isn’t the easiest language to learn - in comparison with other Java-related languages, like Kotlin, which haven’t strayed far from its originator, Scala is almost an attempt to rewrite the rule book. It’s a good multi-purpose programming language that brings together functional programming principles and the object oriented principles you find in Java. It’s also designed for concurrency, giving you a scale of power that just isn’t possible. The one drawback of Scala is that it doesn’t have the consistency in its ecosystem in the way that, say, Java does. This does mean, however, that Scala expertise can be really valuable if you have the time to dedicate time to really getting to know the language. Learn Scala in 2019 if… You’re looking for an alternative to Java that’s more scalable and handles concurrency much better You're working with big data Learn Scala: Learn Scala Programming Professional Scala [Video] Scala Machine Learning Projects Scala Design Patterns - Second Edition       Swift Swift started life as a replacement for Objective-C for iOS developers. While it’s still primarily used by those in the Apple development community, there are some signs that Swift could expand beyond its beginnings to become a language of choice for server and systems programming. The core development team have consistently demonstrated they have a sense of purpose is building a language fit for the future, with versions 3 and 4 both showing significant signs of evolution. Fast, relatively easy to learn, and secure, not only has Swift managed to deliver on its brief to offer a better alternative to Objective-C, it also looks well-suited to many of the challenges programmers will be facing in the years to come. Learn Swift in 2019 if… You want to build apps for Apple products You’re interested in a new way to write server code Learn Swift: Learn Swift by Building Applications Hands-On Full-Stack Development with Swift Hands-On Server-Side Web Development with Swift         Kotlin It makes sense for Kotlin to follow Swift. The parallels between the two are notable; it might be crude, but you could say that Kotlin is to Java what Swift is to Objective-C. There are, of course, some who feel that the comparison isn’t favorable, with accusations that one language is merely copying the other, but perhaps the similarities shouldn’t really be that surprising - they’re both trying to do the same things: provide a better alternative to what already exists. Regardless of the debates, Kotlin is a particularly compelling language if you’re a Java developer. It works extremely well, for example, with Spring Boot to develop web services. Certainly as monolithic Java applications shift into microservices, Kotlin is only going to become more popular. Learn Kotlin in 2019 if… You’re a Java developer that wants to build better apps, faster You want to see what all the fuss is about from the Android community Learn Kotlin: Kotlin Quick Start Guide Learning Kotlin by building Android Applications Learn Kotlin Programming [Video]         C Most of the languages on this list are pretty new, but I’m going to finish with a classic that refuses to go away. C has a reputation for being complicated and hard to learn, but it remains relevant because you can find it in so much of the software we take for granted. It’s the backbone of our operating systems, and used in everyday objects that have software embedded in them. Together, this means C is a language worth learning because it gives you an insight into how software actually works on machines. In a world where abstraction and accessibility rules the software landscape, getting beneath everything can be extremely valuable. Learn C in 2019 if… You’re looking for a new challenge You want to gain a deeper understanding of how software works on your machine You’re interested in developing embedded systems and virtual reality projects Learn C: Learn and Master C Programming For Absolute Beginners! [Video]

At Open Source Technology Summit (OSTS) 2019, Josh Triplett, a Principal Engineer at Intel gave an insight into what Intel is contributing to bring the most loved language, Rust to full parity with C. In his talk titled Intel and Rust: the Future of Systems Programming, he also spoke about the history of systems programming, how C became the “default” systems programming language, what features of Rust gives it an edge over C, and much more. Until now, OSTS was Intel's closed event where the company's business and tech leaders come together to discuss the various trends, technologies, and innovations that will help shape the open-source ecosystem. However, this year was different as the company welcomed non-Intel attendees including media, partners, and developers for the first time. The event hosts keynotes, more than 50 technical sessions, panels, demos covering all the open source technologies Intel is involved in. These include integrated software stacks (edge, AI, infrastructure), firmware, embedded and IoT projects, and cloud system software. This year the event happened from May 14-16 at Stevenson, Washington. What is systems programming Systems programming is the development and management of software that serves as a platform for other software to be built upon. The system software also directly or closely interfaces with computer hardware in order to gain necessary performance and expose abstractions. Unlike application programming where software is created to provide services to the user, it aims to produce software that provides services to the computer hardware. Triplett broadly defines systems programming as “anything that isn't an app.” It includes things like BIOS, firmware, boot loaders, operating systems kernels, embedded and similar types of low-level code, virtual machine implementations. Triplett also counts a web browser as a system software as it is more than “just an app,” they are actually “platforms for websites and web apps,” he says. How C became the “default” systems programming language Previously, most system software including BIOS, boot loaders, and firmware were written in Assembly. In the 1960s, experiments to bring hardware support in high-level languages started, which resulted in the creation of languages such as PL/S, BLISS, BCPL, and extended ALGOL. Then in the 1970s, Dennis Ritchie created the C programming language for the Unix operating system. Derived from the typeless B programming language, C was packed with powerful high-level functionalities and detailed features that were best suited for writing an operating system. Several UNIX components including its kernel were eventually rewritten in C. Many other system software including the Oracle database, a large portion of Windows source code, Linux operating system, were all written in C. C was seeing a huge adoption at this point. But, what exactly made developers comfortable moving to C? Triplett believes that in order to make this move from one language to another, developers have to be comfortable in terms of two things: features and parity. First, the language should offer “sufficiently compelling” features. “It can’t just be a little bit better. It has to be substantially better to warrant the effort and engineering time needed to move,” he adds. As compared to Assembly, C had a lot to offer. It had some degree of type safety, provided portability, better productivity with high-level constructs, and much more readable code. Second, the language has to provide parity, which means developers had to be confident that it is no less capable than Assembly. He states, “It can’t just be better, it also has to be no worse.” In addition to being faster and expressing any type of data that Assembly was able to, it also had what Triplett calls “escape hatch.”  This means you are allowed to make the move incrementally and also combine Assembly if required. Triplett believes that C is now becoming what Assembly was years ago. “C is the new Assembly,” he concludes. Developers are looking for a high-level language that not only addresses the problems in C that can’t be fixed but also leverage other exciting features that these languages provide. Such a language that aims to be compelling enough to make developers move from C should be memory safe, provide automatic memory management,  security, and much more. “Any language that wants to be better than C has to offer a lot more than just protection from buffer overflows if it's actually going to be a compelling alternative. People care about usability and productivity. They care about writing code that is self-explanatory, which accomplishes more work in less code. It also needs to address security issues. Usability and productivity go hand in hand with security. The less code you need to write to accomplish something, the less chance you have of introducing bugs security bugs or otherwise,” he explains. Comparing Rust with C Back in 2006, Graydon Hoare, a Mozilla employee started writing Rust as a personal project. Mozilla, in 2009, started sponsoring the project and also expanded the team to drive further development of the language. One of the reasons why Mozilla got interested is that Firefox was written in more than 4 million lines of C++ code and had quite a bit of highly critical vulnerabilities. Rust was built with safety and concurrency in mind making it the perfect choice for rewriting many components of Firefox under Project Quantum. It is also using Rust to develop Servo, an HTML rendering engine that will eventually replace Firefox’s rendering engine. Many other companies have also started using Rust for their projects including Microsoft, Google, Facebook, Amazon, Dropbox, Fastly, Chef, Baidu, and more. Rust addresses the memory management problem in C. It offers automatic memory management so that developers do not have to manually call free on every object. What sets it apart from other modern languages is that it does not have a garbage collector or runtime system of any kind. Rust instead has the concepts of ownership, borrowing, references, and lifetimes. “Rust has a system of declaring whether any given use of an object is the owner of that object or whether it's just borrowing that object temporarily. If you're just borrowing an object the compiler will keep track of that. It'll make sure that the original sticks around as long as you reference it. Rust makes sure that the owner of the object frees it when it's done and it inserts the call to free at compile time with no extra runtime overhead,” Triplett explains. Not having a runtime is also a plus for Rust. Triplett believes that languages that have a runtime are difficult to use as a system programming language. He adds, “You have to initialize that runtime before you can call any code, you have to use that runtime to call functions, and the runtime itself might run extra code behind your back at unexpected times.” Rust also aims to provide safe concurrent programming. The same features that make it memory safe, keep track of things like which thread own which object, which objects can be passed between threads, and which objects require acquiring locks. These features make Rust compelling enough for developers to choose for systems programming. However, talking about the second criteria, Rust does not have enough parity with C yet. “Achieving parity with C is exactly what got me involved in Rust,” says Triplett Teaching Rust about C compatible unions Triplett's first contribution to the Rust programming language was in the form of the 1444 RFC, which was started in 2015 and got accepted in 2016. This RFC proposed to bring native support for C-compatible unions in Rust that would be defined via a new "contextual keyword" union. Triplett understood the need for this proposal when he wanted to build a virtual machine in Rust and the Linux kernel interface for that /dev/kvm required unions. "I worked with the Rust community and with the language team to get unions into Rust and because of that work I'm actually now part of the Rust language governance team helping to evaluate and guide other changes into the language," he adds. He talked about this RFC in much detail at the very first RustConf in 2016: https://www.youtube.com/watch?v=U8Gl3RTXf88 Support for unnamed struct and union types Another feature that Triplett worked on was the support for unnamed struct and union types in Rust. This has been a widespread C compiler extension for decades and was also included in the C11 standard. This allowed developers to group and layout fields in arbitrary ways to match C data structures used in the Foreign Function Interface (FFI). With this proposal implemented, Rust will be able to represent such types using the same names as the structures without interposing artificial field names that will confuse users of well-established interfaces from existing platforms. A stabilized support for inline Assembly in Rust Systems programming often involves low-level manipulations and requires low-level details of the processors such as privileged instructions. For this, Rust supports using inline Assembly via the ‘asm!’ macro. However, it is only present in the nightly compiler and not yet stabilized. Triplett in a collaboration with other Rust developers is writing a proposal to introduce more robust syntax for inline Assembly. To know more in detail about support for inline Assembly, check out this pre-RFC. BFLOAT16 support into Rust Many Intel processors including Xeon Scalable ‘Cooper Lake-SP’ now support BFLOAT16, a new floating-point format. This truncated 16-bit version of the 32-bit IEEE 754 single-precision floating-point format was mainly designed for deep learning. This format is also used in machine learning libraries like Tensorflow that work with huge datasets. It also makes interoperating with existing systems, functions, and storage much easier. This is why Triplett is working on adding support for BFLOAT16 in Rust so that developers would be able to use the full capabilities of their hardware. FFI/C Parity Working Group This was one of the important announcements that Triplett made. He is starting a working group that will focus on achieving full parity with C. Under this group, he aims to collaborate with both the Rust community and other Intel developers to develop the specifications for the remaining features that need to be implemented in Rust for system programming. This group will also focus on bringing support for systems programming using the stable releases of Rust, not just experimental nightly releases of the compiler. In last week’s Reddit discussion, Triplett shared the current status of the working group, “To pre-answer, one question: the FFI / C Parity working group is in the process of being launched, and hasn't quite kicked off yet. I'll be posting about it here and elsewhere when it is, along with the initial goals.” Watch Josh Triplett’s full OSTS talk to know more about Intel’s contribution to Rust: https://www.youtube.com/watch?v=l9hM0h6IQDo [box type="shadow" align="" class="" width=""]Update: We have made the following corrections based on feedback from Josh Triplett: This year OSTS was open to Intel's partners and press. Previously, the article read 'escape patch', but it is 'escape hatch.' RFC 1444 wasn't last year, it was started in 2015 and accepted in 2016. 'dev KVM' is now corrected to '/dev/kvm'[/box] AMD competes with Intel by launching EPYC Rome, world’s first 7 nm chip for data centers, luring in Twitter and Google Hot Chips 31: IBM Power10, AMD’s AI ambitions, Intel NNP-T, Cerebras largest chip with 1.2 trillion transistors and more Intel’s 10th gen 10nm ‘Ice Lake’ processor offers AI apps, new graphics and best connectivity

This article is an excerpt from the book, Learn LLVM 17 - Second Edition, by Kai Nacke, Amy Kwan. Learn how to build your own compiler, from reading the source to emitting optimized machine code. This book guides you through the JIT compilation framework, extending LLVM in a variety of ways, and using the right tools for troubleshooting.Introduction Generating optimized machine code is a critical task in the compilation process, and the LLVM backend plays a pivotal role in this transformation. The backend translates the LLVM Intermediate Representation (IR), derived from the Abstract Syntax Tree (AST), into machine code that can be executed on target architectures. Understanding how to generate this IR effectively is essential for leveraging LLVM's capabilities. This article delves into the intricacies of generating LLVM IR using a simple expression language example. We'll explore the necessary steps, from declaring library functions to implementing a code generation visitor, ensuring a comprehensive understanding of the LLVM backend's functionality. Generating code with the LLVM backend The task of the backend is to create optimized machine code from the LLVM IR of a module. The IR is the interface to the backend and can be created using a C++ interface or in textual form. Again, the IR is generated from the AST. Textual representation of LLVM IR Before trying to generate the LLVM IR, it should be clear what we want to generate. For our example expression language, the high-level plan is as follows: 1. Ask the user for the value of each variable. 2. Calculate the value of the expression. 3. Print the result. To ask the user to provide a value for a variable and to print the result, two library functions are used: calc_read() and calc_write(). For the with a: 3*a expression, the generated IR is as follows: 1. The library functions must be declared, like in C. The syntax also resembles C. The type before the function name is the return type. The type names surrounded by parenthesis are the argument types. The declaration can appear anywhere in the file: declare i32 @calc_read(ptr) declare void @calc_write(i32) 2. The calc_read() function takes the variable name as a parameter. The following construct defines a constant, holding a and the null byte used as a string terminator in C: @a.str = private constant [2 x i8] c"a\00" 3. It follows the main() function. The parameter names are omitted because they are not used.  Just as in C, the body of the function is enclosed in braces: define i32 @main(i32, ptr) { 4. Each basic block must have a label. Because this is the first basic block of the function, we name it entry: entry: 5. The calc_read() function is called to read the value for the a variable. The nested getelemenptr instruction performs an index calculation to compute the pointer to the first element of the string constant. The function result is assigned to the unnamed %2 variable.  %2 = call i32 @calc_read(ptr @a.str) 6. Next, the variable is multiplied by 3:  %3 = mul nsw i32 3, %2 7. The result is printed on the console via a call to the calc_write() function:  call void @calc_write(i32 %3) 8. Last, the main() function returns 0 to indicate a successful execution:  ret i32 0 } Each value in the LLVM IR is typed, with i32 denoting the 32-bit bit integer type and ptr denoting a pointer. Note: previous versions of LLVM used typed pointers. For example, a pointer to a byte was expressed as i8* in LLVM. Since L LVM 16, opaque pointers are the default. An opaque pointer is just a pointer to memory, without carrying any type information about it. The notation in LLVM IR is ptr.Previous versions of LLVM used typed pointers. For example, a pointer to a byte was expressed as i8* in LLVM. Since L LVM 16, opaque pointers are the default. An opaque pointer is just a pointer to memory, without carrying any type information about it. The notation in LLVM IR is ptr. Since it is now clear what the IR looks like, let’s generate it from the AST.  Generating the IR from the AST The interface, provided in the CodeGen.h header file, is very small:  #ifndef CODEGEN_H #define CODEGEN_H #include "AST.h" class CodeGen { public: void compile(AST *Tree); }; #endifBecause the AST contains the information, the basic idea is to use a visitor to walk the AST. The CodeGen.cpp file is implemented as follows: 1. The required includes are at the top of the file: #include "CodeGen.h" #include "llvm/ADT/StringMap.h" #include "llvm/IR/IRBuilder.h" #include "llvm/IR/LLVMContext.h" #include "llvm/Support/raw_ostream.h" 2. The namespace of the LLVM libraries is used for name lookups: using namespace llvm; 3. First, some private members are declared in the visitor. Each compilation unit is represented in LLVM by the Module class and the visitor has a pointer to the module called M. For easy IR generation, the Builder (of type IRBuilder<>) is used. LLVM has a class hierarchy to represent types in IR. You can look up the instances for basic types such as i32 from the LLVM context. These basic types are used very often. To avoid repeated lookups, we cache the needed type instances: VoidTy, Int32Ty, PtrTy, and Int32Zero. The V member is the current calculated value, which is updated through the tree traversal. And last, nameMap maps a variable name to the value returned from the calc_read() function: namespace { class ToIRVisitor : public ASTVisitor { Module *M; IRBuilder<> Builder; Type *VoidTy; Type *Int32Ty; PointerType *PtrTy; Constant *Int32Zero; Value *V; StringMap<Value *> nameMap;4. The constructor initializes all members: public: ToIRVisitor(Module *M) : M(M), Builder(M->getContext()) { VoidTy = Type::getVoidTy(M->getContext()); Int32Ty = Type::getInt32Ty(M->getContext()); PtrTy = PointerType::getUnqual(M->getContext()); Int32Zero = ConstantInt::get(Int32Ty, 0, true); }5. For each function, a FunctionType instance must be created. In C++ terminology, this is a function prototype. A function itself is defined with a Function instance. The run() method defines the main() function in the LLVM IR first:  void run(AST *Tree) { FunctionType *MainFty = FunctionType::get( Int32Ty, {Int32Ty, PtrTy}, false); Function *MainFn = Function::Create( MainFty, GlobalValue::ExternalLinkage, "main", M); 6. Then we create the BB basic block with the entry label, and attach it to the IR builder: BasicBlock *BB = BasicBlock::Create(M->getContext(), "entry", MainFn); Builder.SetInsertPoint(BB);7. With this preparation done, the tree traversal can begin:     Tree->accept(*this); 8. After the tree traversal, the computed value is printed via a call to the calc_write() function. Again, a function prototype (an instance of FunctionType) has to be created. The only parameter is the current value, V:  FunctionType *CalcWriteFnTy = FunctionType::get(VoidTy, {Int32Ty}, false); Function *CalcWriteFn = Function::Create( CalcWriteFnTy, GlobalValue::ExternalLinkage, "calc_write", M); Builder.CreateCall(CalcWriteFnTy, CalcWriteFn, {V});9. The generation finishes  by returning 0 from the main() function: Builder.CreateRet(Int32Zero); }10. A WithDecl node holds the names of the declared variables. First, we create a function prototype for the calc_read() function:  virtual void visit(WithDecl &Node) override { FunctionType *ReadFty = FunctionType::get(Int32Ty, {PtrTy}, false); Function *ReadFn = Function::Create( ReadFty, GlobalValue::ExternalLinkage, "calc_read", M);11. The method loops through the variable names:  for (auto I = Node.begin(), E = Node.end(); I != E; ++I) { 12. For each  variable, a string  with a variable name is created:  StringRef Var = *I; Constant *StrText = ConstantDataArray::getString( M->getContext(), Var); GlobalVariable *Str = new GlobalVariable( *M, StrText->getType(), /*isConstant=*/true, GlobalValue::PrivateLinkage, StrText, Twine(Var).concat(".str"));13. Then the IR code to call the calc_read() function is created. The string created in the previous step is passed as a parameter:  CallInst *Call = Builder.CreateCall(ReadFty, ReadFn, {Str});14. The returned value is stored in the mapNames map for later use:  nameMap[Var] = Call; }15. The tree traversal continues with the expression:  Node.getExpr()->accept(*this); };16. A Factor node is either a variable name or a number. For a variable name, the value is looked up in the mapNames map. For a number, the value is converted to an integer and turned into a constant value: virtual void visit(Factor &Node) override { if (Node.getKind() == Factor::Ident) { V = nameMap[Node.getVal()]; } else { int intval; Node.getVal().getAsInteger(10, intval); V = ConstantInt::get(Int32Ty, intval, true); } };17. And last, for a BinaryOp node, the right calculation operation must be used:  virtual void visit(BinaryOp &Node) override { Node.getLeft()->accept(*this); Value *Left = V; Node.getRight()->accept(*this); Value *Right = V; switch (Node.getOperator()) { case BinaryOp::Plus: V = Builder.CreateNSWAdd(Left, Right); break; case BinaryOp::Minus: V = Builder.CreateNSWSub(Left, Right); break; case BinaryOp::Mul: V = Builder.CreateNSWMul(Left, Right); break; case BinaryOp::Div: V = Builder.CreateSDiv(Left, Right); break;    }       };       };       }18. With this, the visitor class is complete. The compile() method creates the global context and the  module, runs the tree traversal, and dumps the generated IR to the console:  void CodeGen::compile(AST *Tree) { LLVMContext Ctx; Module *M = new Module("calc.expr", Ctx); ToIRVisitor ToIR(M); ToIR.run(Tree); M->print(outs(), nullptr); }We now have implemented the frontend of the compiler, from reading the source up to generating the IR. Of course, all these components must work together on user input, which is the task of the compiler driver. We also need to implement the functions needed at runtime. Both are topics of the next section covered in the book. Conclusion In conclusion, the process of generating LLVM IR from an AST involves multiple steps, each crucial for producing efficient machine code. This article highlighted the structure and components necessary for this task, including function declarations, basic block management, and tree traversal using a visitor pattern. By carefully managing these elements, developers can harness the power of LLVM to create optimized and reliable machine code. The integration of all these components, alongside user input and runtime functions, completes the frontend implementation of the compiler. This sets the stage for the next phase, focusing on the compiler driver and runtime functions, ensuring seamless execution and integration of the compiled code. Author BioKai Nacke is a professional IT architect currently residing in Toronto, Canada. He holds a diploma in computer science from the Technical University of Dortmund, Germany. and his diploma thesis on universal hash functions was recognized as the best of the semester. With over 20 years of experience in the IT industry, Kai has extensive expertise in the development and architecture of business and enterprise applications. In his current role, he evolves an LLVM/clang-based compiler.nFor several years, Kai served as the maintainer of LDC, the LLVM-based D compiler. He is the author of D Web Development and Learn LLVM 12, both published by Packt. In the past, he was a speaker in the LLVM developer room at the Free and Open Source Software Developers’ European Meeting (FOSDEM).Amy Kwan is a compiler developer currently residing in Toronto, Canada. Originally, from the Canadian prairies, Amy holds a Bachelor of Science in Computer Science from the University of Saskatchewan. In her current role, she leverages LLVM technology as a backend compiler developer. Previously, Amy has been a speaker at the LLVM Developer Conference in 2022 alongside Kai Nacke.

PowerShell is Microsoft’s automation platform for IT Pros. Of late, there have been a lot of questions around the complexity of this latest automation tool by Microsoft. At Microsoft Ignite 2018, Jason Himmelstein, Director of Technical Strategy and Strategic Partnerships, Office Apps & Services MVP, explained the basics of PowerShell and how to truly optimize your SharePoint implementation using this powerful IT pro toolset. While in this post we look at the big picture, you can check out the complete video here: ‘Introduction to PowerShell for the anxious IT pro’. Want to do more with PowerShell? After learning the basics, you can learn how to use PowerShell to automate complex Windows server tasks. You can also improve PowerShell's usability, and control and manage Windows-based environments by working through exciting recipes given in Windows Server 2019 Automation with PowerShell Cookbook - Third Edition written by Thomas Lee.  Himmelstein starts off by saying PowerShell isn’t a packaged executable, nor it is developer-centric that needs one to understand code, and it is easy for an IT Pro to understand. What is PowerShell? Windows PowerShell is Microsoft’s task automation framework, consisting of a command-line shell and associated scripting language built on .NET Framework. It provides full access to COM and WMI, enabling administrators to perform administrative tasks on both local and remote Windows systems. In simple words, PowerShell is an object-based, not a text-based, command-line interface for Microsoft Technologies. This means results in PowerShell can be acted upon and not just read from. One can cause huge damage to an environment using PowerShell as there is no back button in PowerShell. However, to check what must have gone wrong, you can check the logs but can not undo actions. Why PowerShell matters Regardless of the platform a person uses such as Office 365, Azure, etc., PowerShell can be easily implemented due to its cross-platform capability. Himmelstein also highlights one can also get started with Azure PowerShell by trying it out in an Azure Cloud Shell environment, an interactive, authenticated, browser-accessible shell for managing Azure resources.  Azure Cloud Shell comes equipped with commonly used CLI tools including Linux shell interpreters, PowerShell modules, Azure tools, text editors, source control, build tools, container tools, database tools and more. Cloud Shell also includes language support for several popular programming languages such as Node.js, .NET and Python. Cloud Shell also securely authenticates automatically for instant access to your resources through the Azure CLI or Azure PowerShell cmdlets. Users can use PowerShell in Cloud Shell. One can also develop applications using PowerShell or can use PowerShell via Source Control Management (SCM). Basics of PowerShell PowerShell Hardware There are two ways one can use PowerShell; one is via the PowerShell Console, which is similar to a command line. The other is PowerShell ISE (Integrated Scripting Environment). One thing Himmelstein encourages is, “we run PowerShell in the Console and we write PowerShell in the ISE.” The reason is there are certain functionalities that do not work in the ISE when one hits the ‘Run’ command. In such cases, the user will have to take that PowerShell out, copy it, save the file and run it in a command window. cmdlets Cmdlets are the main building blocks of PowerShell. These are mini commands that perform one action. These have the ability to pipe the output of one cmdlet into further cmdlets. These can also perform equality tests with expressions such as -eq, -lt, -match; one can diff easily within a PowerShell. Modules There are four types of Modules in PowerShell: Script: A Script module is a file (.psm1) that contains any valid Windows PowerShell code. Binary: A binary module is a .NET framework assembly (.dll) that contains compiled code. Manifest: A module Manifest is a Windows PowerShell data file (.psd1) that describes the contents of a module and determines how a module is processed. Dynamic: A dynamic module does not persist to disk. It is created using New Module, is intended to be short-lived, and cannot be accessed by Get-Module. Himmelstein prefers not to use the Dynamic module as it persists for just one session. Objects and Members Objects are instances of classes and have properties and methods. Members are properties and methods of an object. Properties define what an Object is and Methods define what you can do with the object. Himmelstein puts together all these terms in a simple way: Objects = stuff Cmdlets = things you can do with the stuff Modules = list of things you can do with the stuff Properties = details about the stuff Methods = instructions for things you can do with the stuff PipeLine Using PipeLines one can chain objects together for processing. The output of a pipelined object becomes the object itself. Functional Explanation Get-command: Gets all the cmdlet installed on your computer. Get-help: Displays additional information about a cmdlet Get-member: Listing the Properties and Methods of a Command or Object Get-verb: Gets approved Windows PowerShell verbs Start-transcript: Logs everything you do in that PowerShell window to a file Get- history: If you didn’t start transcript, you can still review your history before closing your Shell or ISE window. Tips for PowerShell beginners Use Variables: You can use any variables except the ones that are reserved by the system, which you will be prompted when you try to enter a reserved variable. Call one thing at a time Comment your scripts as this may save you a lot of time. Create scripts using an ISE/IDE, you can also use the Visual Studio Code and then execute in Shell. Dispose of your objects. Close the command window by typing Exit. Test before using in Production Write reusable scripts. What Powershell beginners should avoid Rewriting your variables Hard coding your scripts such as Password as it may get fired by PowerShell Taking code from the internet or vendor and just Run in your environment (You should read every code before you run it in your environment). Assuming the code is not harmful; it is. There is no back button in PowerShell and you cannot undo things. Running your code in an IDE/ISE and expect everything to work. PowerShell Syntax and Bracketology Syntax ‘#’ is for Comment ‘+’ is for Add ‘=’, ‘-eq’, are for Equal ‘!’, ‘-ne’, ‘-not’ are for ‘not equal’ Brackets ‘()’ Curved brackets also known as Parentheses are used for required options, compulsory arguments, or control structures. ‘{}’ Curly brackets are used for block expression within a command block and is also used to open a code block ‘[]’ Square brackets are used to denote optional elements or parameters and also used for match functions. Now that you know the basics of PowerShell, you can start performing key admin tasks on Windows Server 2019. To further learn how to employ best practices for writing PowerShell scripts and configuring Windows Server 2019 and leverage PowerShell to automate complex Windows server tasks, check out our book, Windows Server 2019 Automation with PowerShell Cookbook - Third Edition written by Thomas Lee. Weaponizing PowerShell with Metasploit and how to defend against PowerShell attacks [Tutorial] Scripting with Windows Powershell Desired State Configuration [Video] Automate tasks using Azure PowerShell and Azure CLI [Tutorial]

In this article by Vikram Murugesan, the author of the book Microservices Deployment Cookbook, we will see a brief introduction to concept of the microservices. (For more resources related to this topic, see here.) Writing microservices with Spring Boot Now that our project is ready, let's look at how to write our microservice. There are several Java-based frameworks that let you create microservices. One of the most popular frameworks from the Spring ecosystem is the Spring Boot framework. In this article, we will look at how to create a simple microservice application using Spring Boot. Getting ready Any application requires an entry point to start the application. For Java-based applications, you can write a class that has the main method and run that class as a Java application. Similarly, Spring Boot requires a simple Java class with the main method to run it as a Spring Boot application (microservice). Before you start writing your Spring Boot microservice, you will also require some Maven dependencies in your pom.xml file. How to do it… Create a Java class called com.packt.microservices.geolocation.GeoLocationApplication.java and give it an empty main method: package com.packt.microservices.geolocation; public class GeoLocationApplication { public static void main(String[] args) { // left empty intentionally } } Now that we have our basic template project, let's make our project a child project of Spring Boot's spring-boot-starter-parent pom module. This module has a lot of prerequisite configurations in its pom.xml file, thereby reducing the amount of boilerplate code in our pom.xml file. At the time of writing this, 1.3.6.RELEASE was the most recent version: <parent> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-parent</artifactId> <version>1.3.6.RELEASE</version> </parent> After this step, you might want to run a Maven update on your project as you have added a new parent module. If you see any warnings about the version of the maven-compiler plugin, you can either ignore it or just remove the <version>3.5.1</version> element. If you remove the version element, please perform a Maven update afterward. Spring Boot has the ability to enable or disable Spring modules such as Spring MVC, Spring Data, and Spring Caching. In our use case, we will be creating some REST APIs to consume the geolocation information of the users. So we will need Spring MVC. Add the following dependencies to your pom.xml file: <dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> </dependencies> We also need to expose the APIs using web servers such as Tomcat, Jetty, or Undertow. Spring Boot has an in-memory Tomcat server that starts up as soon as you start your Spring Boot application. So we already have an in-memory Tomcat server that we could utilize. Now let's modify the GeoLocationApplication.java class to make it a Spring Boot application: package com.packt.microservices.geolocation; import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; @SpringBootApplication public class GeoLocationApplication { public static void main(String[] args) { SpringApplication.run(GeoLocationApplication.class, args); } } As you can see, we have added an annotation, @SpringBootApplication, to our class. The @SpringBootApplication annotation reduces the number of lines of code written by adding the following three annotations implicitly: @Configuration @ComponentScan @EnableAutoConfiguration If you are familiar with Spring, you will already know what the first two annotations do. @EnableAutoConfiguration is the only annotation that is part of Spring Boot. The AutoConfiguration package has an intelligent mechanism that guesses the configuration of your application and automatically configures the beans that you will likely need in your code. You can also see that we have added one more line to the main method, which actually tells Spring Boot the class that will be used to start this application. In our case, it is GeoLocationApplication.class. If you would like to add more initialization logic to your application, such as setting up the database or setting up your cache, feel free to add it here. Now that our Spring Boot application is all set to run, let's see how to run our microservice. Right-click on GeoLocationApplication.java from Package Explorer, select Run As, and then select Spring Boot App. You can also choose Java Application instead of Spring Boot App. Both the options ultimately do the same thing. You should see something like this on your STS console: If you look closely at the console logs, you will notice that Tomcat is being started on port number 8080. In order to make sure our Tomcat server is listening, let's run a simple curl command. cURL is a command-line utility available on most Unix and Mac systems. For Windows, use tools such as Cygwin or even Postman. Postman is a Google Chrome extension that gives you the ability to send and receive HTTP requests. For simplicity, we will use cURL. Execute the following command on your terminal: curl http://localhost:8080 This should give us an output like this: {"timestamp":1467420963000,"status":404,"error":"Not Found","message":"No message available","path":"/"} This error message is being produced by Spring. This verifies that our Spring Boot microservice is ready to start building on with more features. There are more configurations that are needed for Spring Boot, which we will perform later in this article along with Spring MVC. Writing microservices with WildFly Swarm WildFly Swarm is a J2EE application packaging framework from RedHat that utilizes the in-memory Undertow server to deploy microservices. In this article, we will create the same GeoLocation API using WildFly Swarm and JAX-RS. To avoid confusion and dependency conflicts in our project, we will create the WildFly Swarm microservice as its own Maven project. This article is just here to help you get started on WildFly Swarm. When you are building your production-level application, it is your choice to either use Spring Boot, WildFly Swarm, Dropwizard, or SparkJava based on your needs. Getting ready Similar to how we created the Spring Boot Maven project, create a Maven WAR module with the groupId com.packt.microservices and name/artifactId geolocation-wildfly. Feel free to use either your IDE or the command line. Be aware that some IDEs complain about a missing web.xml file. We will see how to fix that in the next section. How to do it… Before we set up the WildFly Swarm project, we have to fix the missing web.xml error. The error message says that Maven expects to see a web.xml file in your project as it is a WAR module, but this file is missing in your project. In order to fix this, we have to add and configure maven-war-plugin. Add the following code snippet to your pom.xml file's project section: <build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-war-plugin</artifactId> <version>2.6</version> <configuration> <failOnMissingWebXml>false</failOnMissingWebXml> </configuration> </plugin> </plugins> </build> After adding the snippet, save your pom.xml file and perform a Maven update. Also, if you see that your project is using a Java version other than 1.8. Again, perform a Maven update for the changes to take effect. Now, let's add the dependencies required for this project. As we know that we will be exposing our APIs, we have to add the JAX-RS library. JAX-RS is the standard JSR-compliant API for creating RESTful web services. JBoss has its own version of JAX-RS. So let's  add that dependency to the pom.xml file: <dependencies> <dependency> <groupId>org.jboss.spec.javax.ws.rs</groupId> <artifactId>jboss-jaxrs-api_2.0_spec</artifactId> <version>1.0.0.Final</version> <scope>provided</scope> </dependency> </dependencies> The one thing that you have to note here is the provided scope. The provide scope in general means that this JAR need not be bundled with the final artifact when it is built. Usually, the dependencies with provided scope will be available to your application either via your web server or application server. In this case, when Wildfly Swarm bundles your app and runs it on the in-memory Undertow server, your server will already have this dependency. The next step toward creating the GeoLocation API using Wildfly Swarm is creating the domain object. Use the com.packt.microservices.geolocation.GeoLocation.java file. Now that we have the domain object, there are two classes that you need to create in order to write your first JAX-RS web service. The first of those is the Application class. The Application class in JAX-RS is used to define the various components that you will be using in your application. It can also hold some metadata about your application, such as your basePath (or ApplicationPath) to all resources listed in this Application class. In this case, we are going to use /geolocation as our basePath. Let's see how that looks: package com.packt.microservices.geolocation; import javax.ws.rs.ApplicationPath; import javax.ws.rs.core.Application; @ApplicationPath("/geolocation") public class GeoLocationApplication extends Application { public GeoLocationApplication() {} } There are two things to note in this class; one is the Application class and the other is the @ApplicationPath annotation—both of which we've already talked about. Now let's move on to the resource class, which is responsible for exposing the APIs. If you are familiar with Spring MVC, you can compare Resource classes to Controllers. They are responsible for defining the API for any specific resource. The annotations are slightly different from that of Spring MVC. Let's create a new resource class called com.packt.microservices.geolocation.GeoLocationResource.java that exposes a simple GET API: package com.packt.microservices.geolocation; import java.util.ArrayList; import java.util.List; import javax.ws.rs.GET; import javax.ws.rs.Path; import javax.ws.rs.Produces; @Path("/") public class GeoLocationResource { @GET @Produces("application/json") public List<GeoLocation> findAll() { return new ArrayList<>(); } } All the three annotations, @GET, @Path, and @Produces, are pretty self explanatory. Before we start writing the APIs and the service class, let's test the application from the command line to make sure it works as expected. With the current implementation, any GET request sent to the /geolocation URL should return an empty JSON array. So far, we have created the RESTful APIs using JAX-RS. It's just another JAX-RS project: In order to make it a microservice using Wildfly Swarm, all you have to do is add the wildfly-swarm-plugin to the Maven pom.xml file. This plugin will be tied to the package phase of the build so that whenever the package goal is triggered, the plugin will create an uber JAR with all required dependencies. An uber JAR is just a fat JAR that has all dependencies bundled inside itself. It also deploys our application in an in-memory Undertow server. Add the following snippet to the plugins section of the pom.xml file: <plugin> <groupId>org.wildfly.swarm</groupId> <artifactId>wildfly-swarm-plugin</artifactId> <version>1.0.0.Final</version> <executions> <execution> <id>package</id> <goals> <goal>package</goal> </goals> </execution> </executions> </plugin> Now execute the mvn clean package command from the project's root directory, and wait for the Maven build to be successful. If you look at the logs, you can see that wildfly-swarm-plugin will create the uber JAR, which has all its dependencies. You should see something like this in your console logs: After the build is successful, you will find two artifacts in the target directory of your project. The geolocation-wildfly-0.0.1-SNAPSHOT.war file is the final WAR created by the maven-war-plugin. The geolocation-wildfly-0.0.1-SNAPSHOT-swarm.jar file is the uber JAR created by the wildfly-swarm-plugin. Execute the following command in the same terminal to start your microservice: java –jar target/geolocation-wildfly-0.0.1-SNAPSHOT-swarm.jar After executing this command, you will see that Undertow has started on port number 8080, exposing the geolocation resource we created. You will see something like this: Execute the following cURL command in a separate terminal window to make sure our API is exposed. The response of the command should be [], indicating there are no geolocations: curl http://localhost:8080/geolocation Now let's build the service class and finish the APIs that we started. For simplicity purposes, we are going to store the geolocations in a collection in the service class itself. In a real-time scenario, you will be writing repository classes or DAOs that talk to the database that holds your geolocations. Get the com.packt.microservices.geolocation.GeoLocationService.java interface. We'll use the same interface here. Create a new class called com.packt.microservices.geolocation.GeoLocationServiceImpl.java that extends the GeoLocationService interface: package com.packt.microservices.geolocation; import java.util.ArrayList; import java.util.Collections; import java.util.List; public class GeoLocationServiceImpl implements GeoLocationService { private static List<GeoLocation> geolocations = new ArrayList<>(); @Override public GeoLocation create(GeoLocation geolocation) { geolocations.add(geolocation); return geolocation; } @Override public List<GeoLocation> findAll() { return Collections.unmodifiableList(geolocations); } } Now that our service classes are implemented, let's finish building the APIs. We already have a very basic stubbed-out GET API. Let's just introduce the service class to the resource class and call the findAll method. Similarly, let's use the service's create method for POST API calls. Add the following snippet to GeoLocationResource.java: private GeoLocationService service = new GeoLocationServiceImpl(); @GET @Produces("application/json") public List<GeoLocation> findAll() { return service.findAll(); } @POST @Produces("application/json") @Consumes("application/json") public GeoLocation create(GeoLocation geolocation) { return service.create(geolocation); } We are now ready to test our application. Go ahead and build your application. After the build is successful, run your microservice: let's try to create two geolocations using the POST API and later try to retrieve them using the GET method. Execute the following cURL commands in your terminal one by one: curl -H "Content-Type: application/json" -X POST -d '{"timestamp": 1468203975, "userId": "f1196aac-470e-11e6-beb8-9e71128cae77", "latitude": 41.803488, "longitude": -88.144040}' http://localhost:8080/geolocation This should give you something like the following output (pretty-printed for readability): { "latitude": 41.803488, "longitude": -88.14404, "userId": "f1196aac-470e-11e6-beb8-9e71128cae77", "timestamp": 1468203975 } curl -H "Content-Type: application/json" -X POST -d '{"timestamp": 1468203975, "userId": "f1196aac-470e-11e6-beb8-9e71128cae77", "latitude": 9.568012, "longitude": 77.962444}' http://localhost:8080/geolocation This command should give you an output similar to the following (pretty-printed for readability): { "latitude": 9.568012, "longitude": 77.962444, "userId": "f1196aac-470e-11e6-beb8-9e71128cae77", "timestamp": 1468203975 } To verify whether your entities were stored correctly, execute the following cURL command: curl http://localhost:8080/geolocation This should give you an output like this (pretty-printed for readability): [ { "latitude": 41.803488, "longitude": -88.14404, "userId": "f1196aac-470e-11e6-beb8-9e71128cae77", "timestamp": 1468203975 }, { "latitude": 9.568012, "longitude": 77.962444, "userId": "f1196aac-470e-11e6-beb8-9e71128cae77", "timestamp": 1468203975 } ] Whatever we have seen so far will give you a head start in building microservices with WildFly Swarm. Of course, there are tons of features that WildFly Swarm offers. Feel free to try them out based on your application needs. I strongly recommend going through the WildFly Swarm documentation for any advanced usages. Writing microservices with Dropwizard Dropwizard is a collection of libraries that help you build powerful applications quickly and easily. The libraries vary from Jackson, Jersey, Jetty, and so on. You can take a look at the full list of libraries on their website. This ecosystem of libraries that help you build powerful applications could be utilized to create microservices as well. As we saw earlier, it utilizes Jetty to expose its services. In this article, we will create the same GeoLocation API using Dropwizard and Jersey. To avoid confusion and dependency conflicts in our project, we will create the Dropwizard microservice as its own Maven project. This article is just here to help you get started with Dropwizard. When you are building your production-level application, it is your choice to either use Spring Boot, WildFly Swarm, Dropwizard, or SparkJava based on your needs. Getting ready Similar to how we created other Maven projects,  create a Maven JAR module with the groupId com.packt.microservices and name/artifactId geolocation-dropwizard. Feel free to use either your IDE or the command line. After the project is created, if you see that your project is using a Java version other than 1.8. Perform a Maven update for the change to take effect. How to do it… The first thing that you will need is the dropwizard-core Maven dependency. Add the following snippet to your project's pom.xml file: <dependencies> <dependency> <groupId>io.dropwizard</groupId> <artifactId>dropwizard-core</artifactId> <version>0.9.3</version> </dependency> </dependencies> Guess what? This is the only dependency you will need to spin up a simple Jersey-based Dropwizard microservice. Before we start configuring Dropwizard, we have to create the domain object, service class, and resource class: com.packt.microservices.geolocation.GeoLocation.java com.packt.microservices.geolocation.GeoLocationService.java com.packt.microservices.geolocation.GeoLocationImpl.java com.packt.microservices.geolocation.GeoLocationResource.java Let's see what each of these classes does. The GeoLocation.java class is our domain object that holds the geolocation information. The GeoLocationService.java class defines our interface, which is then implemented by the GeoLocationServiceImpl.java class. If you take a look at the GeoLocationServiceImpl.java class, we are using a simple collection to store the GeoLocation domain objects. In a real-time scenario, you will be persisting these objects in a database. But to keep it simple, we will not go that far. To be consistent with the previous, let's change the path of GeoLocationResource to /geolocation. To do so, replace @Path("/") with @Path("/geolocation") on line number 11 of the GeoLocationResource.java class. We have now created the service classes, domain object, and resource class. Let's configure Dropwizard. In order to make your project a microservice, you have to do two things: Create a Dropwizard configuration class. This is used to store any meta-information or resource information that your application will need during runtime, such as DB connection, Jetty server, logging, and metrics configurations. These configurations are ideally stored in a YAML file, which will them be mapped to your Configuration class using Jackson. In this application, we are not going to use the YAML configuration as it is out of scope for this article. If you would like to know more about configuring Dropwizard, refer to their Getting Started documentation page at http://www.dropwizard.io/0.7.1/docs/getting-started.html. Let's  create an empty Configuration class called GeoLocationConfiguration.java: package com.packt.microservices.geolocation; import io.dropwizard.Configuration; public class GeoLocationConfiguration extends Configuration { } The YAML configuration file has a lot to offer. Take a look at a sample YAML file from Dropwizard's Getting Started documentation page to learn more. The name of the YAML file is usually derived from the name of your microservice. The microservice name is usually identified by the return value of the overridden method public String getName() in your Application class. Now let's create the GeoLocationApplication.java application class: package com.packt.microservices.geolocation; import io.dropwizard.Application; import io.dropwizard.setup.Environment; public class GeoLocationApplication extends Application<GeoLocationConfiguration> { public static void main(String[] args) throws Exception { new GeoLocationApplication().run(args); } @Override public void run(GeoLocationConfiguration config, Environment env) throws Exception { env.jersey().register(new GeoLocationResource()); } } There are a lot of things going on here. Let's look at them one by one. Firstly, this class extends Application with the GeoLocationConfiguration generic. This clearly makes an instance of your GeoLocationConfiguraiton.java class available so that you have access to all the properties you have defined in your YAML file at the same time mapped in the Configuration class. The next one is the run method. The run method takes two arguments: your configuration and environment. The Environment instance is a wrapper to other library-specific objects such as MetricsRegistry, HealthCheckRegistry, and JerseyEnvironment. For example, we could register our Jersey resources using the JerseyEnvironment instance. The env.jersey().register(new GeoLocationResource())line does exactly that. The main method is pretty straight-forward. All it does is call the run method. Before we can start the microservice, we have to configure this project to create a runnable uber JAR. Uber JARs are just fat JARs that bundle their dependencies in themselves. For this purpose, we will be using the maven-shade-plugin. Add the following snippet to the build section of the pom.xml file. If this is your first plugin, you might want to wrap it in a <plugins> element under <build>: <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-shade-plugin</artifactId> <version>2.3</version> <configuration> <createDependencyReducedPom>true</createDependencyReducedPom> <filters> <filter> <artifact>*:*</artifact> <excludes> <exclude>META-INF/*.SF</exclude> <exclude>META-INF/*.DSA</exclude> <exclude>META-INF/*.RSA</exclude> </excludes> </filter> </filters> </configuration> <executions> <execution> <phase>package</phase> <goals> <goal>shade</goal> </goals> <configuration> <transformers> <transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer" /> <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer"> <mainClass>com.packt.microservices.geolocation.GeoLocationApplication</mainClass> </transformer> </transformers> </configuration> </execution> </executions> </plugin> The previous snippet does the following: It creates a runnable uber JAR that has a reduced pom.xml file that does not include the dependencies that are added to the uber JAR. To learn more about this property, take a look at the documentation of maven-shade-plugin. It utilizes com.packt.microservices.geolocation.GeoLocationApplication as the class whose main method will be invoked when this JAR is executed. This is done by updating the MANIFEST file. It excludes all signatures from signed JARs. This is required to avoid security errors. Now that our project is properly configured, let's try to build and run it from the command line. To build the project, execute mvn clean package from the project's root directory in your terminal. This will create your final JAR in the target directory. Execute the following command to start your microservice: java -jar target/geolocation-dropwizard-0.0.1-SNAPSHOT.jar server The server argument instructs Dropwizard to start the Jetty server. After you issue the command, you should be able to see that Dropwizard has started the in-memory Jetty server on port 8080. If you see any warnings about health checks, ignore them. Your console logs should look something like this: We are now ready to test our application. Let's try to create two geolocations using the POST API and later try to retrieve them using the GET method. Execute the following cURL commands in your terminal one by one: curl -H "Content-Type: application/json" -X POST -d '{"timestamp": 1468203975, "userId": "f1196aac-470e-11e6-beb8-9e71128cae77", "latitude": 41.803488, "longitude": -88.144040}' http://localhost:8080/geolocation This should give you an output similar to the following (pretty-printed for readability): { "latitude": 41.803488, "longitude": -88.14404, "userId": "f1196aac-470e-11e6-beb8-9e71128cae77", "timestamp": 1468203975 } curl -H "Content-Type: application/json" -X POST -d '{"timestamp": 1468203975, "userId": "f1196aac-470e-11e6-beb8-9e71128cae77", "latitude": 9.568012, "longitude": 77.962444}' http://localhost:8080/geolocation This should give you an output like this (pretty-printed for readability): { "latitude": 9.568012, "longitude": 77.962444, "userId": "f1196aac-470e-11e6-beb8-9e71128cae77", "timestamp": 1468203975 } To verify whether your entities were stored correctly, execute the following cURL command: curl http://localhost:8080/geolocation It should give you an output similar to the following (pretty-printed for readability): [ { "latitude": 41.803488, "longitude": -88.14404, "userId": "f1196aac-470e-11e6-beb8-9e71128cae77", "timestamp": 1468203975 }, { "latitude": 9.568012, "longitude": 77.962444, "userId": "f1196aac-470e-11e6-beb8-9e71128cae77", "timestamp": 1468203975 } ] Excellent! You have created your first microservice with Dropwizard. Dropwizard offers more than what we have seen so far. Some of it is out of scope for this article. I believe the metrics API that Dropwizard uses could be used in any type of application. Writing your Dockerfile So far in this article, we have seen how to package our application and how to install Docker. Now that we have our JAR artifact and Docker set up, let's see how to Dockerize our microservice application using Docker. Getting ready In order to Dockerize our application, we will have to tell Docker how our image is going to look. This is exactly the purpose of a Dockerfile. A Dockerfile has its own syntax (or Dockerfile instructions) and will be used by Docker to create images. Throughout this article, we will try to understand some of the most commonly used Dockerfile instructions as we write our Dockerfile for the geolocation tracker microservice. How to do it… First, open your STS IDE and create a new file called Dockerfile in the geolocation project. The first line of the Dockerfile is always the FROM instruction followed by the base image that you would like to create your image from. There are thousands of images on Docker Hub to choose from. In our case, we would need something that already has Java installed on it. There are some images that are official, meaning they are well documented and maintained. Docker Official Repositories are very well documented, and they follow best practices and standards. Docker has its own team to maintain these repositories. This is essential in order to keep the repository clear, thus helping the user make the right choice of repository. To read more about Docker Official Repositories, take a look at https://docs.docker.com/docker-hub/official_repos/ We will be using the Java official repository. To find the official repository, go to hub.docker.com and search for java. You have to choose the one that says official. At the time of writing this, the Java image documentation says it will soon be deprecated in favor of the openjdk image. So the first line of our Dockerfile will look like this: FROM openjdk:8 As you can see, we have used version (or tag) 8 for our image. If you are wondering what type of operating system this image uses, take a look at the Dockerfile of this image, which you can get from the Docker Hub page. Docker images are usually tagged with the version of the software they are written for. That way, it is easy for users to pick from. The next step is creating a directory for our project where we will store our JAR artifact. Add this as your next line: RUN mkdir -p /opt/packt/geolocation This is a simple Unix command that creates the /opt/packt/geolocation directory. The –p flag instructs it to create the intermediate directories if they don't exist. Now let's create an instruction that will add the JAR file that was created in your local machine into the container at /opt/packt/geolocation. ADD target/geolocation-0.0.1-SNAPSHOT.jar /opt/packt/geolocation/ As you can see, we are picking up the uber JAR from target directory and dropping it into the /opt/packt/geolocation directory of the container. Take a look at the / at the end of the target path. That says that the JAR has to be copied into the directory. Before we can start the application, there is one thing we have to do, that is, expose the ports that we would like to be mapped to the Docker host ports. In our case, the in-memory Tomcat instance is running on port 8080. In order to be able to map port 8080 of our container to any port to our Docker host, we have to expose it first. For that, we will use the EXPOSE instruction. Add the following line to your Dockerfile: EXPOSE 8080 Now that we are ready to start the app, let's go ahead and tell Docker how to start a container for this image. For that, we will use the CMD instruction: CMD ["java", "-jar", "/opt/packt/geolocation/geolocation-0.0.1-SNAPSHOT.jar"] There are two things we have to note here. Once is the way we are starting the application and the other is how the command is broken down into comma-separated Strings. First, let's talk about how we start the application. You might be wondering why we haven't used the mvn spring-boot:run command to start the application. Keep in mind that this command will be executed inside the container, and our container does not have Maven installed, only OpenJDK 8. If you would like to use the maven command, take that as an exercise, and try to install Maven on your container and use the mvn command to start the application. Now that we know we have Java installed, we are issuing a very simple java –jar command to run the JAR. In fact, the Spring Boot Maven plugin internally issues the same command. The next thing is how the command has been broken down into comma-separated Strings. This is a standard that the CMD instruction follows. To keep it simple, keep in mind that for whatever command you would like to run upon running the container, just break it down into comma-separated Strings (in whitespaces). Your final Dockerfile should look something like this: FROM openjdk:8 RUN mkdir -p /opt/packt/geolocation ADD target/geolocation-0.0.1-SNAPSHOT.jar /opt/packt/geolocation/ EXPOSE 8080 CMD ["java", "-jar", "/opt/packt/geolocation/geolocation-0.0.1-SNAPSHOT.jar"] This Dockerfile is one of the simplest implementations. Dockerfiles can sometimes get bigger due to the fact that you need a lot of customizations to your image. In such cases, it is a good idea to break it down into multiple images that can be reused and maintained separately. There are some best practices to follow whenever you create your own Dockerfile and image. Though we haven't covered that here as it is out of the scope of this article, you still should take a look at and follow them. To learn more about the various Dockerfile instructions, go to https://docs.docker.com/engine/reference/builder/. Building your Docker image We created the Dockerfile, which will be used in this article to create an image for our microservice. If you are wondering why we would need an image, it is the only way we can ship our software to any system. Once you have your image created and uploaded to a common repository, it will be easier to pull your image from any location. Getting ready Before you jump right into it, it might be a good idea to get yourself familiar with some of the most commonly used Docker commands. In this article, we will use the build command. Take a look at this URL to understand the other commands: https://docs.docker.com/engine/reference/commandline/#/image-commands. After familiarizing yourself with the commands, open up a new terminal, and change your directory to the root of the geolocation project. Make sure your docker-machine instance is running. If it is not running, use the docker-machine start command to run your docker-machine instance: docker-machine start default If you have to configure your shell for the default Docker machine, go ahead and execute the following command: eval $(docker-machine env default) How to do it… From the terminal, issue the following docker build command: docker build –t packt/geolocation. We'll try to understand the command later. For now, let's see what happens after you issue the preceding command. You should see Docker downloading the openjdk image from Docker Hub. Once the image has been downloaded, you will see that Docker tries to validate each and every instruction provided in the Dockerfile. When the last instruction has been processed, you will see a message saying Successfully built. This says that your image has been successfully built. Now let's try to understand the command. There are three things to note here: The first thing is the docker build command itself. The docker build command is used to build a Docker image from a Dockerfile. It needs at least one input, which is usually the location of the Dockerfile. Dockerfiles can be renamed to something other than Dockerfile and can be referred to using the –f option of the docker build command. An instance of this being used is when teams have different Dockerfiles for different build environments, for example, using DockerfileDev for the dev environment, DockerfileStaging for the staging environment, and DockerfileProd for the production environment. It is still encouraged as best practice to use other Docker options in order to keep the same Dockerfile for all environments. The second thing is the –t option. The –t option takes the name of the repo and a tag. In our case, we have not mentioned the tag, so by default, it will pick up latest as the tag. If you look at the repo name, it is different from the official openjdk image name. It has two parts: packt and geolocation. It is always a good practice to put the Docker Hub account name followed by the actual image name as the name of your repo. For now, we will use packt as our account name, we will see how to create our own Docker Hub account and use that account name here. The third thing is the dot at the end. The dot operator says that the Dockerfile is located in the current directory, or the present working directory to be more precise. Let's go ahead and verify whether our image was created. In order to do that, issue the following command on your terminal: docker images The docker images command is used to list down all images available in your Docker host. After issuing the command, you should see something like this: As you can see, the newly built image is listed as packt/geolocation in your Docker host. The tag for this image is latest as we did not specify any. The image ID uniquely identifies your image. Note the size of the image. It is a few megabytes bigger than the openjdk:8 image. That is most probably because of the size of our executable uber JAR inside the container. Now that we know how to build an image using an existing Dockerfile, we are at the end of this article. This is just a very quick intro to the docker build command. There are more options that you can provide to the command, such as CPUs and memory. To learn more about the docker build command, take a look at this page: https://docs.docker.com/engine/reference/commandline/build/ Running your microservice as a Docker container We successfully created our Docker image in the Docker host. Keep in mind that if you are using Windows or Mac, your Docker host is the VirtualBox VM and not your local computer. In this article, we will look at how to spin off a container for the newly created image. Getting ready To spin off a new container for our packt/geolocation image, we will use the docker run command. This command is used to run any command inside your container, given the image. Open your terminal and go to the root of the geolocation project. If you have to start your Docker machine instance, do so using the docker-machine start command, and set the environment using the docker-machine env command. How to do it… Go ahead and issue the following command on your terminal: docker run packt/geolocation Right after you run the command, you should see something like this: Yay! We can see that our microservice is running as a Docker container. But wait—there is more to it. Let's see how we can access our microservice's in-memory Tomcat instance. Try to run a curl command to see if our app is up and running: Open a new terminal instance and execute the following cURL command in that shell: curl -H "Content-Type: application/json" -X POST -d '{"timestamp": 1468203975, "userId": "f1196aac-470e-11e6-beb8-9e71128cae77", "latitude": 41.803488, "longitude": -88.144040}' http://localhost:8080/geolocation Did you get an error message like this? curl: (7) Failed to connect to localhost port 8080: Connection refused Let's try to understand what happened here. Why would we get a connection refused error when our microservice logs clearly say that it is running on port 8080? Yes, you guessed it right: the microservice is not running on your local computer; it is actually running inside the container, which in turn is running inside your Docker host. Here, your Docker host is the VirtualBox VM called default. So we have to replace localhost with the IP of the container. But getting the IP of the container is not straightforward. That is the reason we are going to map port 8080 of the container to the same port on the VM. This mapping will make sure that any request made to port 8080 on the VM will be forwarded to port 8080 of the container. Now go to the shell that is currently running your container, and stop your container. Usually, Ctrl + C will do the job. After your container is stopped, issue the following command: docker run –p 8080:8080 packt/geolocation The –p option does the port mapping from Docker host to container. The port number to the left of the colon indicates the port number of the Docker host, and the port number to the right of the colon indicates that of the container. In our case, both of them are same. After you execute the previous command, you should see the same logs that you saw before. We are not done yet. We still have to find the IP that we have to use to hit our RESTful endpoint. The IP that we have to use is the IP of our Docker Machine VM. To find the IP of the docker-machine instance, execute the following command in a new terminal instance: docker-machine ip default. This should give you the IP of the VM. Let's say the IP that you received was 192.168.99.100. Now, replace localhost in your cURL command with this IP, and execute the cURL command again: curl -H "Content-Type: application/json" -X POST -d '{"timestamp": 1468203975, "userId": "f1196aac-470e-11e6-beb8-9e71128cae77", "latitude": 41.803488, "longitude": -88.144040}' http://192.168.99.100:8080/geolocation This should give you an output similar to the following (pretty-printed for readability): { "latitude": 41.803488, "longitude": -88.14404, "userId": "f1196aac-470e-11e6-beb8-9e71128cae77", "timestamp": 1468203975 } This confirms that you are able to access your microservice from the outside. Take a moment to understand how the port mapping is done. The following figure shows how your machine, VM, and container are orchestrated: This confirms that you are able to access your microservice from the outside. Summary We looked at an example of a geolocation tracker application to see how it can be broken down into smaller and manageable services. Next, we saw how to create the GeoLocationTracker service using the Spring Boot framework. Resources for Article: Further resources on this subject: Domain-Driven Design [article] Breaking into Microservices Architecture [article] A capability model for microservices [article]

You’ve probably heard that Facebook, Twitter, Google, Microsoft, and other tech industry behemoths keep their entire codebase, all services, applications and tools in a single huge repository - a monorepo. If you’re used to the standard way most teams manage their codebase - one application, service or tool per repository - this sounds very strange. Many people conclude it must only solve problems the likes of Google and Facebook have. This is a guest post by Viktor Charypar, Technical Director at Red Badger. But monorepos are not only useful if you can build a custom version control system to cope. They actually have many advantages even at a smaller scale that standard tools like Git handle just fine. Using a monorepo can result in fewer barriers in the software development lifecycle. It can allow faster feedback loops, less time spent looking for code,, and less time reporting bugs and waiting for them to be fixed. It also makes it much easier to analyze a huge treasure trove of interesting data about how your software is actually built and where problem areas are. We’ve used a monorepo at one of our clients for almost three years and it’s been great. I really don’t see why you wouldn’t. But roughly every two months I tend to have a conversation with someone who’s not used to working in this way and the entire idea just seems totally crazy to them. And the conversation tends to always follow the same path, starting with the sheer size and quickly moving on to dependency management, testing and versioning strategies. It gets complicated. It’s time I finally wrote down a coherent reasoning behind why I believe monorepos should be the default way we manage a codebase. Especially if you’re building something even vaguely microservices based, you have multiple teams and want to share common code. What do you mean “just one repo”? Just so we’re all thinking about the same thing, when I say monorepo, I’m talking about a strategy of storing all the code you as an organization are responsible for. This could be a project, a programme of work, or the entirety of a product and infrastructure code of your company in a single repository, under one revision history. Individual components (libraries, services, custom tools, infrastructure automation, ...) are stored alongside each other in folders. It’s analogous to the UNIX file tree which has a single root, as opposed to multiple, device based roots in Windows operating systems. People not familiar with the concept typically have a fairly strong reaction to the idea. One giant repo? Why would anyone do that? That cannot possibly scale! Many different objections come out, most of them often only tangentially related to storing all the code together. Occasionally, people get almost religious about it (I am talking about engineers after all). Despite being used by some of the largest tech companies, it is a relatively foreign concept and on the surface goes against everything you’ve been taught about not making huge monolithic things. It also seems like we’re fixing things that are not broken: everyone in the world is doing multiple repos, building and sharing artifacts (npm modules, JARs, ruby gems…), using SemVer to version and manage dependencies and long running branches to patch bugs in older versions of code, right? Surely if it’s industry standard it must be the right thing to do. Well, I don’t believe so. I personally think almost every single one of those things is harder, more laborious, more brittle, harder to test and generally just more wasteful than the equivalent approach you get as a consequence of a monorepo. And a few of the capabilities a monorepo enables can’t be replicated in a multi-repo situation even if you build a lot of infrastructure around it, basically because you introduce distributed computing problems and get on the bad side of CAP theorem (we’ll look at this closer below). Apart from letting you make dependency management easier and testing more reliable than it can get with multiple repos, monorepo will also give you a few simple, important, but easy to underestimate advantages. The biggest advantages of using a monorepo It's easier to find and discover your code in a monorepo With a monorepo, there is no question about where all the code is and when you have access to some of it, you can see all of it. It may come as a surprise, but making code visible to the rest of the organization isn’t always the default behavior. Human insecurities get in the way and people create private repositories and squirrel code away to experiment with things “until they are ready”. Typically, when the thing does get “ready”, it now has a Continuous Integration (CI) service attached, many hyperlinks lead to it from emails, chat rooms and internal wikis, several people have cloned the repo and it’s now quite a hassle to move the code to a more visible, obvious place and so it stays where it started. As a consequence, it is often quite hard work to find all the code for the project and gain access to it, which is hard and expensive for new joiners and hurts collaboration in general. You could say this is a matter of discipline and I will agree with you, but why leave to individual discipline what you can simply prevent by telling everyone that all the code belongs in the one repo, it’s completely ok to put even little experiments and pet projects there. You never know what they will grow into and putting them in the repo has basically no cost attached. Visibility aids understanding of how to use internal APIs (internal in the sense of being designed and built by your organisation). The ability to search the entire codebase from within your editor and find usages of the call you’re considering to use is very powerful. Code editors and languages can also be set up for cross-references to work, which means you can follow references into shared libraries and find usages of shared code across the codebase. And I mean the entire codebase. This also enables all kind of analyses to be done on the codebase and its history. Knowing the totality of the codebase and having a history of all the code lets you see dependencies, find parts of the codebase only committed to by a very limited group of people, hotspots changing suspiciously frequently or by a large number of people… Your codebase is the source of truth about what your engineering organization is producing, it contains an incredible amount of interesting information we typically just ignore. Monorepos give you more flexibility when moving code Conway’s Law famously states that “organizations which design systems (...) are constrained to produce designs which are copies of the communication structures of these organisations”. This is due to the level of communication necessary to produce a coherent piece of software. The further away in the organisation an owner of a piece of software is, the harder it is to directly influence it, so you design strict interfaces to insulate yourself from the effect of “their” changes. This typically affects the repository structure as well. There are two problems with this: the structure is generally chosen upfront, before we know what the right shape of the software is, and changing the structure has a cost attached. With each service and library being in a separate repository, the module boundaries are quite a lot stronger than if they are all in one repository. Extracting common pieces of code into a shared library becomes more difficult and involves a setup of a whole new repository - full with CI integration, pull request templates and labels, access control setup… hard work. In a monorepo, these boundaries are much more fluid and flexible: moving code between services and libraries, extracting new ones or inlining libraries back into their consumers all become as easy as general refactoring. There is no reason to use a completely different set of tools to change the small-scale and the large-scale structure of your codebase. The only real downside is tooling support for access control and declaring ownership. However, as monorepos get more popular, this support is getting better. GitHub now supports codeowners, for example. We will get there. A monorepo gives you a single history timeline While visibility and flexibility are quite convenient, the one feature of a monorepo which is very hard (if not impossible) to replicate is the single history timeline. We’ll go into why it’s so hard further below, but for now let’s look at the advantages it brings. Single history timelines gives us a reliable total order of changes to the codebase over time. This means that for any two contributions to the codebase, we can definitively and reliably decide which came first and which came second. It should never be ambiguous. It also means that each commit in a monorepo is a snapshot of the system as it was at that given moment. This enables a really interesting capability: it means cross-cutting changes can be made atomically, safely, in one go. Atomic cross-cutting commits Atomic cross-cutting commits make two specific scenarios much easier to achieve. First, externally forced global migrations are much easier and quicker. Let’s say multiple services use a common database and need the password and we need to rotate it. The password itself is (hopefully!) stored in a secure credential store, but at least the reference to it will be in several different places within the codebase. If the reference changes (let’s say the reference is generated every time), we can update every specific the mention of it at once, in one commit, with a search & replac. This will get everything working again. Second, and more important, we can change APIs and update both producer and all consumers at the same time, atomically. For example, we can add an endpoint to an API service and migrate consumers to use the new endpoint. In the next commit, we can remove the old API endpoint as it’s no longer needed. If you're trying to do this across multiple repositories with their own histories, the change will have to be split into several parallel commits. This leaves the potential for the two changes to overlap and happen in the wrong order. Some consumers can get migrated, then the endpoint gets removed, then the rest of the consumers get migrated. The mid-stage is an inconsistent state and an attempt to use the not-yet-migrated consumers will fail attempting to call an API endpoint that no longer exists. Monorepos remove inconsistencies in your dependencies Inconsistencies between dependent modules are the entire reason why dependency management and versioning exist. In a monorepo, the above scenario simply can’t happen. And that’s how the conversation about storing code in one place ends up being about versioning and dependency management. Monorepos essentially make the problem go away (which is my favourite kind of problem solving). Okay, this isn’t entirely true. There are still consequences to making breaking changes to APIs. For one, you need to update all the consumers, which is work, but you also need to build all of them, test everything works and deploy it all. This is quite hard for (micro)services that get individually deployed: making coordinated deployment of multiple services atomic is possible but not trivial. You can use a blue-green strategy, for example, to make sure there is no moment in time where only some services changed but not others. It gets harder for shared libraries. Building and publishing artifacts of new versions and updating all consumers to use the new version are still at least two commits, otherwise you’d be referring to versions that won’t exist until the builds finish. Now, things are getting inconsistent again and the view of what is supposed to work together is getting blurred in time again. And what if someone sneaks some changes in between the two commits. We are, once again, in a race. Unless... Building from the latest source Yes. What if instead of building and publishing shared code as prebuilt artifacts (binaries, jars, gems, npm modules), we build each deployable service completely from source. Every time a service changes, it is entirely rebuilt, including all dependencies. This is a fair bit of work for some compiled languages. However, it can be optimised with incremental build tools which skip work that’s already been done and cached. Some, like Go, solve it by simply having designed for a fast compiler. For dynamic languages, it’s just a matter of setting up include paths correctly and bundling all the relevant code. The added benefit here is you don’t need to do anything special if you’re working on a set of interdependent projects locally. No more `npm link`. The more interesting consequence is how this affects changing a shared dependency. When building from source, you have to make sure every time that happens, all the consumers get rebuilt using it. This is great, everyone gets the latest and greatest things immediately. ...right? Don’t worry, I can hear the alarm bells ringing in your head all the way from here. Your service depends on tens, if not hundreds of libraries. Any time anyone makes a mistake and breaks any of them, it breaks your code? Hell no. But hear me out. This is a problem of understanding dependencies and testing consumers. The important consequence of building from source is you now have a single handle on what is supposed to work together. There are no separate versions, you know what to test and it’s just one thing at any one time. Push dependency management In manual dependency update management - I will call it “pull” dependency management - you as a consumer are responsible for updating your dependencies as you see fit and making sure everything still works. If you find a bug, you simply don’t upgrade. Instead you report the bug to the maintainer and expect them to fix it. This can be months after the bug was introduced and the bug may have already been fixed in a newer version you haven’t yet upgraded to because things have moved on quite a bit while you were busy hitting a deadline and it would now be a sizable investment to upgrade. Now you’re a little stuck and all the ways out are a significant amount of work for someone, all that because the feedback loop is too long. Normally, as a library maintainer, you’re never quite certain how to make sure you’re not breaking anything. Even if you could run your consumers’ test suites, which consumers at what versions do you test against? And as a DevOps team doing 24/7 support for a system, how do you know which version or versions of a library is used across your services. What do you need to update to roll out that important bug fix to your customers? In push dependency management, quite a few things are the other way round. As a consumer, you’re not responsible for updating, it is done for you - effectively, you depended on the “latest” version of everything. Every time a maintainer of the library makes a change you are responsible for testing for regressions. No, not manually! You do have unit tests, right? Right?? Please have a solid regression test suite you trust, it’s 2019. So with your unit test suite, all you need to do is run it. Actually no, let’s let the maintainer run it. If they introduce a problem, they get immediate feedback from you, before their change ever hits the master branch. And this is the contract agreement in push dependency management: If you make a change and break anyone, you are responsible for fixing them. They are responsible for supplying a good enough, by their own standard, automated mechanism for you to verify things still work. The definition of “works” is the tests pass. Seriously though, you need to have a decent regression test suite! Continuous integration for push dependencies: the final piece of the monorepo puzzle The main missing piece of tooling around monorepos is support for push dependencies in CI systems. It’s quite straightforward to implement the strategy yourself, but it’s still hard enough to be worth some shared tooling. Unfortunately, the existing build tools geared towards monorepos like Bazel and Buck take over the entire build process from more familiar tools (like Maven or Babel) and you need to switch to them. Although to be fair, in exchange, you get very performant incremental builds. A lighter tooling, which lets you express dependencies between components in a monorepo, in a language agnostic way, and is only responsible for deciding which build jobs needs to be triggered given a set of changed components in a commit seems to be missing. So I built one. It’s far from perfect, but it should do the trick. Hopefully, someone with more time on their hands will eventually come up with something similarly cheap to introduce into your build system and the community will adopt it more widely. The main takeaway is that if we build from source in a monorepo we can set up a central Continuous Integration system responsible for triggering builds for all projects potentially affected by a change, intended to make sure you didn’t break anything with the work you did, whether it belongs to you or someone else. This is next to impossible in a multi-repo codebase because of the blurriness of history mentioned above. It’s interesting to me that we have this problem today in the larger ecosystem. Everywhere. And we stumble forward and somewhat successfully live with upstream changes occasionally breaking us, because we don’t really have a better choice. We don’t have the ability to test all the consumers in the world and fix things when we break them. But if we can, at least for our own codebase, why wouldn’t we do that? Along with a “you broke it you fix it” policy. Building from source in a monorepo allows that. It also makes it significantly easier to make breaking changes much harder to make. That said... About breaking changes There are two kinds of changes that break the consumer - the ones you introduce by accident, intending the keep backwards compatibility, and then the intentional ones. The first kind should not be too laborious to fix: once you find out what’s wrong, fix it in one place, make sure you didn’t break anything else, done. The second kind is harder. If you absolutely have to make an intentional breaking change, you will need to update all the consumers. Yes. That’s the deal. And that’s also fair. I’m not sure why we’re okay with breaking changes being intentionally introduced upstream on a whim. In any other area of human endeavour, a breach of contract makes people angry and they will expect you to make good by them. Yet we accept breaking changes in software as a fact of life. “It’s fine, I bumped the major version!” Semantic versioning: a bad idea It’s not fine. In fact, semantic versioning is just a bad idea. I know that’s a bold claim and this is a whole separate article (which I promise to write soon), but I’ll try to do it at least some justice here. Semantic versioning is meant to convey some meaning with the version number, but the chosen meanings it expresses are completely arbitrary. First of all, semver only talks about API contract, not behaviour. Adding side effects or changing performance characteristics of an API call for worse while keeping the data interface the same is a completely legal patch level change. And I bet you consider that a breaking change, because it will break your app. Second, does anyone really care about minor vs. patch? The promise is the API doesn’t break. So really we only care about major or other. Major is a dealbreaker, otherwise we’re ok. From a consumer perspective a major version bump spells trouble and potentially a lot of work. Making breaking changes is a mean thing to do to your consumers and you can and should avoid them. Just keep the old API around and working and add the new one next to it. As for version numbers, the most important meaning to convey seems to be “how old?” because code tends to rot, and so versioning by date might be a good choice. But, you say, I’ll have more and more code to maintain! Well yes, of course. And that’s the other problem with semver, the expectation that even old versions still get patches and fixes. It’s not very explicitly stated but it’s there. And because we typically maintain old versions on long-running branches in version control, it’s not even very visible in the codebase. What if you kept older APIs around, but deprecated them and the answer to bugs in them would be to migrate to the newer version of a particular call, which doesn’t have the bug? Would you care about having that old code? It just sits there in the codebase, until nobody uses it. It would also be much less work for the consumer, it’s just one particular call. Also, the bug is typically deeper inside your code, so it’s actually more likely you can fix it in one go for all the API surfaces, old or new. Doing the same thing in the branching model is N times the work (for N maintenance branches). There are technologies that follow this model out of necessity. One example is GraphQL, which was built to solve (among other things) the problem of many old API consumers in people’s hands and the need to support all of them for at least some time. In GraphQL, you deprecate data fields in your API and they become invisible in documentation and introspection calls, but they still work as they used to. Possibly forever. Or at least until barely anyone uses them. The other option you have if you want to keep an older version of a library around and maintain it in a monorepo is to make a copy of the folder and work on the two separately. It’s the same thing as cutting a long running branch, you’re just making a copy in “file space” not “branch space”. And it’s more visible and representative of reality - both versions exist as first-class components being maintained. There are many different versioning and maintenance strategies you could adopt, but in my opinion the preference should be to invest effort into the latest version, making breaking changes only when absolutely inevitable (and at that point, isn’t the new version just a new thing? Like Luxon, the next version of Moment.js) and making updates trivial for your consumers. And if it’s trivial you can do it for them. Ultimately it was your decision to break the API, so you should also do the work, it’s only fair and it makes you evaluate the cost-benefit trade-off of the change. In a monorepo with building from source, this versioning strategy happens naturally. You can, however, adopt others. You just lose some of the guarantees and make feedback loops longer. Versioning strategy is really an orthogonal concept to storing code in a single repository, but the relative costs do change if you use one. Versioning by using a single version that cuts across the system becomes a lot cheaper, which means breaking changes becomes more expensive. This tends to lead to more discussions about versioning. This is actually true for most of the things we covered above. You can, but don’t have to adopt these strategies with a monorepo. Pay as you go monorepos It’s totally possible to just store everything in a single repo and not do anything else. You’ll get the visibility of what exists and flexibility of boundaries and ownership. You can still publish individual build artifacts and pull-manage dependencies (but you’d be missing out). Add building from source and you get the single snapshot benefit - you now know what code runs in a particular version of your system and to an extent, you can think about it as a monolith, despite being formed of many different independent modules and services. Add dependency aware continuous integration and the feedback loop around issues introduced while working on the codebase gets much much shorter, allowing you to go faster and waste less time on carefully managing versions, reporting bugs, making big forklift upgrades, etc. Things tend to get out of control much less. It’s simpler. Best of all, you can mix and match strategies. If you have a hugely popular library in your monorepo and each change in it triggers a build of hundreds of consumers, it only takes a couple of those builds being flakey and it will make it very hard to get builds for the changes in the library to pass. This is really a CI problem to fix (and there are so many interesting strategies out there), but sometimes you can’t do that easily. You could also say the feedback loop is now too tight for the scale and start versioning the library’s intentional releases. This still doesn’t mean you have to publish versioned artifacts. You can have a stable copy of the library in the repo, which consumers depend on, and a development copy next to it which the maintainers work on. Releasing then means moving changes from the development folder to the release one and getting its builds to pass. Or, if you wish, you can publish artifacts and let consumers pull them on their own time and report bugs to you. And you still don’t need to promise fixes for older versions without upgrading to the latest (Libraries should really have a published “code of maintenance” outlining the promises and setting maintenance expectations). And if you have to, I would again recommend making a copy, not branching. In fact, in a monorepo, branching might just not be a very good idea. Temporary branches are still useful to work on proposed changes, but long-running branches just hide the full truth about the system. And so does relying on old commits. The copies of code being used exist either way, the are still relevant and you still need to consider them for testing and security patching, they are just hidden in less apparent dimensions of the codebase “space” - branch dimension or time dimension. These are hard to think about and visualise, so maybe it’s not a good idea to use them to keep relevant and current versions of the code and stick to them as change proposal and “time travel” mechanisms. Hopefully you can see that there’s an entire spectrum of strategies you can follow but don’t have to adopt wholesale. I’m sold, but... can’t we do all this with a multi-repo? Most of the things discussed above are not really strictly dependent on monorepos, they are more a natural consequence of adopting one. You can follow versioning strategies other than semver outside of a monorepo. You can probably implement an automated version bumping system which will upgrade all the dependents of a library and test them, logging issues if they don’t pass. What you can’t do outside of a monorepo, as far as I can tell, is the atomic snapshotting of history to have a clear view of the system. And have the same view of the system a year ago. And be able to reproduce it. As soon as multiple parallel version histories are established, this ability goes away and you introduce distributed state. It’s impossible to update all the “heads” in this multi history at the same time, consistently. In version control, like git, the histories are ordered by the “follows” relationship. Later version follows - points to - its predecessor. To get a consistent, canonical view of time, there needs to be a single entrypoint. Without this central entry point, it’s impossible to define a consistent order across the entire set, it depends on where we start looking. Essentially, you already chose Partition from the three CAP properties. Now you can pick either Consistency or Availability. Typically, availability is important and so you lose consistency. You could choose consistency instead, but that would mean you can’t have availability - in order to get a consistent snapshot of the state of all of the repos, write access would need to be stopped while the snapshot is taken. In a monorepo, you don’t have partitioning, and can therefore have consistency and availability. From a physics perspective, multiple repositories with their own history effectively create a kind of spacetime, where each repository is a place and references across repos represent information propagating across space. The speed of that propagation isn’t infinite - it’s not instant. If changes happen in two places close enough in time, from the perspective of those two places, they happen in a globally inconsistent order, first the local change, then the remote change. Neither of the views is better and more true and it’s impossible to decide which of the changes came first. Unless, that is, we introduce an agreed upon central point which references all the repositories that exist and every time one of them updates, the reference in this master gets updated and a revision is created. Congratulations, we created a monorepo, well done us. The benefits of going all-in when it comes to monorepos As I said at the beginning, adopting the monorepo approach fully will result in fewer barriers in the software development lifecycle. You get faster feedback loops - the ability to test consumers of libraries before checking in a change and immediate feedback. You will spend less time looking for code and working out how it gets assembled together. You won’t need to set up repositories or ask for permissions to contribute. You can spend more time-solving problems to help your customers instead of problems you created for yourself. It takes some time to get the tooling setup right, but you only do it once, all the later projects get the setup for free. Some of the tooling is a little lacking, but in our experience there are no show stoppers. A stable, reliable CI is an absolute must, but that’s regardless of monorepos. Monorepos should also help make builds repeatable. The repo does eventually get big, but it takes years and years and hundreds of people to get to a size where it actually becomes a real problem. The Linux kernel is a monorepo and it’s probably still at least an order of magnitude bigger than your project (it is bigger than ours anyway, despite having hundreds of engineers involved at this point). Basically, you’re not Google or Microsoft. And when you are, you’ll be able to afford optimising your version control system. The UX of your code review and source hosting tooling is probably the first thing that will break, not the underlying infrastructure. For smoother scaling, the one recommendation I have is to set a file size limit - accidentally committed large files are quite hard to remove, at least in git. After using a monorepo for over two years we’re still yet to have any big technical issues with it (plenty of political ones, but that’s a story for another day) and we see the same benefits as Google reported in their recent paper. I honestly don’t really know why you would start with any other strategy. Viktor Charypar is Technical Director at Red badger, a digital consultancy based in London. You can follow him on Twitter @charypar or read more of his writing on the Red Badger blog here.