Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Save more on your purchases! discount-offer-chevron-icon
Savings automatically calculated. No voucher code required.
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Newsletter Hub
Free Learning
Arrow right icon
timer SALE ENDS IN
0 Days
:
00 Hours
:
00 Minutes
:
00 Seconds

Author Posts

117 Articles
article-image-how-to-face-a-critical-rag-driven-generative-ai-challenge
Mr. Denis Rothman
06 Nov 2024
15 min read
Save for later

How to Face a Critical RAG-driven Generative AI Challenge

Mr. Denis Rothman
06 Nov 2024
15 min read
This article is an excerpt from the book, "RAG-Driven Generative AI", by Denis Rothman. Explore the transformative potential of RAG-driven LLMs, computer vision, and generative AI with this comprehensive guide, from basics to building a complex RAG pipeline.IntroductionOn a bright Monday morning, Dakota sits down to get to work and is called by the CEO of their software company, who looks quite worried. An important fire department needs a conversational AI agent to train hundreds of rookie firefighters nationwide on drone technology. The CEO looks dismayed because the data provided is spread over many websites around the country. Worse, the management of the fire department is coming over at 2 PM to see a demonstration to decide whether to work with Dakata’s company or not. Dakota is smiling. The CEO is puzzled.  Dakota explains that the AI team can put a prototype together in a few hours and be more than ready by 2 PM and get to work. The strategy is to divide the AI team into three sub-teams that will work in parallel on three pipelines based on the reference Deep Lake, LlamaIndex and OpenAI RAG program* they had tested and approved a few weeks back.  Pipeline 1: Collecting and preparing the documents provided by the fire department for this Proof of Concept(POC). Pipeline 2: Creating and populating a Deep Lake vector store with the first batch of documents while the Pipeline 1 team continues to retrieve and prepare the documents. Pipeline 3: Indexed-based RAG with LlamaIndex’s integrated OpenAI LLM performed on the first batch of vectorized documents. The team gets to work at around 9:30 AM after devising their strategy. The Pipeline 1 team begins by fetching and cleaning a batch of documents. They run Python functions to remove punctuation except for periods and noisy references within the content. Leveraging the automated functions they already have through the educational program, the result is satisfactory.  By 10 AM, the Pipeline 2 team sees the first batch of documents appear on their file server. They run the code they got from the RAG program* to create a Deep Lake vector store and seamlessly populate it with an OpenAI embedding model, as shown in the following excerpt: from llama_index.core import StorageContext vector_store_path = "hub://denis76/drone_v2" dataset_path = "hub://denis76/drone_v2" # overwrite=True will overwrite dataset, False will append it vector_store = DeepLakeVectorStore(dataset_path=dataset_path, overwrite=True)  Note that the path of the dataset points to the online Deep Lake vector store. The fact that the vector store is serverless is a huge advantage because there is no need to manage its size, storage process and just begin to populate it in a few seconds! Also, to process the first batch of documents, overwrite=True, will force the system to write the initial data. Then, starting the second batch,  the Pipeline 2 team can run overwrite=False, to append the following documents. Finally,  LlamaIndex automatically creates a vector store index: storage_context = StorageContext.from_defaults(vector_store=vector_store) # Create an index over the documents index = VectorStoreIndex.from_documents(documents, storage_context=storage_context) By 10:30 AM, the Pipeline 3 team can visualize the vectorized(embedded) dataset in their Deep Lake vector store. They create a LlamaIndex query engine on the dataset: from llama_index.core import VectorStoreIndex vector_store_index = VectorStoreIndex.from_documents(documents) … vector_query_engine = vector_store_index.as_query_engine(similarity_top_k=k, temperature=temp, num_output=mt) Note that the OpenAI Large Language Model is seamlessly integrated into LlamaIndex with the following parameters: k, in this case, k=3, specifies the number of documents to retrieve from the vector store. The retrieval is based on the similarity of embedded user inputs and embedded vectors within the dataset. temp, in this case temp=0.1, determines the randomness of the output. A low value such as 0.1 forces the similarity search to be precise. A higher value would allow for more diverse responses, which we do not want for this technological conversational agent. mt, in this case, mt=1024, determines the maximum number of tokens in the output. A cosine similarity function was added to make sure that the outputs matched the sample user inputs: from sentence_transformers import SentenceTransformer model = SentenceTransformer('all-MiniLM-L6-v2') def calculate_cosine_similarity_with_embeddings(text1, text2):     embeddings1 = model.encode(text1)     embeddings2 = model.encode(text2)     similarity = cosine_similarity([embeddings1], [embeddings2])     return similarity[0][0] By 11:00 AM, all three pipeline teams are warmed up and ready to go full throttle! While the Pipeline 2 team was creating the vector store and populating it with the first batch of documents, the Pipeline 1 team prepared the next several batches. At 11:00 AM, Dakota gave the green light to run all three pipelines simultaneously. Within a few minutes, the whole RAG-driven generative AI system was humming like a beehive! By 1:00 PM, Dakota and the three pipeline teams were working on a PowerPoint slideshow with a copilot. Within a few minutes, it was automatically generated based on their scenario. At 1:30 PM, they had time to grab a quick lunch. At 2:00 pm, the fire department management, Dakota’s team, and the CEO of their software company were in the meeting room.  Dakota’s team ran the PowerPoint slide show and began the demonstration with a simple input:  user_input="Explain how drones employ real-time image processing and machine learning algorithms to accurately detect events in various environmental conditions." The response displayed was satisfactory: Drones utilize real-time image processing and machine learning algorithms to accurately detect events in various environmental conditions by analyzing data captured by their sensors and cameras. This technology allows drones to process visual information quickly and efficiently, enabling them to identify specific objects, patterns, or changes in the environment in real-time. By employing these advanced algorithms, drones can effectively monitor and respond to different situations, such as wildfires, wildlife surveys, disaster relief efforts, and agricultural monitoring with precision and accuracy. Dakota’s team then showed that the program could track and display the original documents the response was based on. At one point, the fire department’s top manager, Taylor, exclaimed, “Wow, this is impressive! It’s exactly what we were looking for! " Of course, Dakato’s CEO began discussing the number of users, cost, and timelines with Taylor. In the meantime, Dakota and the rest of the fire department’s team went out to drink some coffee and get to know each other. Fire departments intervene at short notice efficiently for emergencies. So can expert-level AI teams! https://github.com/Denis2054/RAG-Driven-Generative-AI/blob/main/Chapter03/Deep_Lake_LlamaIndex_OpenAI_RAG.ipynb ConclusionIn facing a high-stakes, time-sensitive challenge, Dakota and their AI team demonstrated the power and efficiency of RAG-driven generative AI. By leveraging a structured, multi-pipeline approach with tools like Deep Lake, LlamaIndex, and OpenAI’s advanced models, the team was able to integrate scattered data sources quickly and effectively, delivering a sophisticated, real-time conversational AI prototype tailored for firefighter training on drone technology. Their success showcases how expert planning, resourceful use of AI tools, and teamwork can transform a complex project into a streamlined solution that meets client needs. This case underscores the potential of generative AI to create responsive, practical solutions for critical industries, setting a new standard for rapid, high-quality AI deployment in real-world applications.Author Bio Denis Rothman graduated from Sorbonne University and Paris-Diderot University, and as a student, he wrote and registered a patent for one of the earliest word2vector embeddings and word piece tokenization solutions. He started a company focused on deploying AI and went on to author one of the first AI cognitive NLP chatbots, applied as a language teaching tool for Mo�t et Chandon (part of LVMH) and more. Denis rapidly became an expert in explainable AI, incorporating interpretable, acceptance-based explanation data and interfaces into solutions implemented for major corporate projects in the aerospace, apparel, and supply chain sectors. His core belief is that you only really know something once you have taught somebody how to do it.
Read more
  • 0
  • 0
  • 26932

article-image-why-become-an-advanced-salesforce-administrator-enrico-murru-salesforce-mvp-solution-and-technical-architect-interview
Fatema Patrawala
14 Nov 2019
12 min read
Save for later

Why become an advanced Salesforce administrator: Enrico Murru, Salesforce MVP, Solution and Technical Architect [Interview]

Fatema Patrawala
14 Nov 2019
12 min read
As per a recent IDC study, the forecast for new jobs demanding Salesforce skills shows a huge surge from last year. The numbers reveal that the demand is set to create 3.3 million jobs in the Salesforce ecosystem by 2022.  Additionally, among Indeed’s top 10 best jobs include Salesforce-specific, Salesforce Administrator ranking 4th and Salesforce Developer ranking at 6th place. Though Salesforce admins are not developers, but they create easy-to-use dashboards, intelligent workflows and applications for any project. They keep the Salesforce users happy and business processes smart, hence they are high in demand. Companies, especially in the US, know the potential and value Salesforce admins bring and are making serious human capital investments. We recently interviewed, Enrico Murru, a Solution and Technical Architect, a platinum Salesforce partner and Salesforce MVP to discuss the Salesforce ecosystem, his Salesforce expert journey, various certifications for Salesforce admins, and how they enhance their careers. Enrico is the author of the latest edition of our book, Salesforce Advanced Administrator Guide. This guide extends beyond being an administrator certification and covers advanced platform features and functions such as configuration, automation, security, and customization. It is packed with exam-oriented questions and mock tests to help you earn advanced administrator credentials. On the Salesforce ecosystem and Enrico’s journey to becoming a Salesforce MVP As per a recent 10K Advisors research, the Salesforce ecosystem is innovating faster than the talent can keep pace. This has resulted in great career opportunities but also introduced challenges for Salesforce end-users. How is Salesforce dealing with the challenges? How can administrators and developers leverage growth opportunities in Salesforce? When I started working with Salesforce about 10 years ago, I had never heard about the Salesforce ecosystem in my life: honestly Italy was not a hot market at that time, that’s why my (small at the time) company had a chance to work with big customers...we were among the few Salesforce system integrators in our whole country, after all. About 4 to 5 years ago things changed dramatically and Italy finally aligned with the rest of the world: Salesforce was in high demand among all kinds of companies (small or huge, no difference). The Italian market is one of the fastest growing; we started growing more and more due to increasing number of customers joining us but we started suffering from lack of professionals. We built an internal academy but it wasn’t enough, we still needed (and currently need) more developers, administrators and business analysts, the demand has exceeded the supply! The amount of “free to access documentation” is huge, the Salesforce Ohana has produced tons of content with blogs, webinars and tens of books. When Salesforce delivered Trailhead to the world we all had a boost in training: learning Salesforce became ever easier! No surprise the number of people getting certified has increased drastically, and it’s not uncommon now to see people with 5, 10 or 20 certifications on their career backpack: you don’t need to stay hours and hours with your head in a book, now you can learn 15 minutes a time when you are free between your working tasks. This is a HUGE revolution: learn a bit often and you keep yourself always on the trail, for free! From now on, anyone can become a Salesforce trailblazer and start building their trail: a lot of people have decided to change jobs and dipped into the Salesforce world with few to no experience in computer science. However when it’s time to get a certification, especially when it is your first certification, Trailhead is not enough: you need some real-world experience (no Trailhead can prepare you enough, experience is an amazing fuel for increasing your overall knowledge). A book can be a good compromise to boost your knowledge while giving you the right amount of experience that the author melt on each topic, and that’s why I chose to start this amazing trail with Packt: I wanted to do something I’ve never done before (writing a book) while delivering then Ohana more chances to pass a certification...I guess this is a win-win situation! How did you start your journey of becoming a Salesforce expert? Did being a Java developer, help you in some way? What motivated you to make the choice? Good question and the answer is that I have to thank the randomness that we can encounter daily on our lives (we can call it destiny, if you prefer). I started working as a Java developer (I came from an Electronic Engineering MSc) for a small company in my local town (Cagliari, Italy). After a while I got bored of what I was doing (boredom is a fuel for me) then I decided to move to Ireland. I got immediately the day after I landed in Cork a new job with a great income (compared to what I was earning in Italy)...but I was not 100% sure if I wanted to move abroad and that’s why I rejected that position and got back to Italy (some say it was an act of cowardice, I partially agree but I was not ready to change my life so much at that time). After just 2 months from my return home, my boss told me about a new opportunity: moving to northern Italy to join WebResults, a small company (we were just 15 people, including the CEO and CTO) that worked with something called “Salesforce”. I accepted the challenge and moved for 6 months with my spouse-to-be to WebResults headquarters: I discovered the world of Salesforce and I immediately fell in love with it. In a few weeks I learnt all that I needed to start my journey as a Salesforce developer. Years to come, I’m still working with WebResults (that in the meanwhile has been acquired by Engineering Spa, the greatest Italian consultant company) as a Salesforce Solution and Technical Architect (the amount of time I spend on coding at work has dramatically dropped unfortunately) and with the honorable Salesforce MVP title I try to evangelize my company and all the Salesforce Ohana buddies anyway I can! So if you ask me if my Java dev position helped me to arrive where I am, the answer is “definitely yes” but there is a lot more in the story! On various Salesforce certifications and why he wrote a book There are many certifications available for beginners as well as for experienced CRM developers. How should one go about choosing them? How do different Salesforce certification programs enhance a developer’s career? If you want to start your journey with Salesforce you have to choose primarily among the following paths (more details at https://trailhead.salesforce.com/credentials/administratoroverview, but you can build your own trail!): Administrator Developer Marketer Consultant Architect In my experience any aspiring Salesforce consultant should start from bases, even though she is a skilled business analysts with 20 years of experience: you need to know how the Salesforce Lightning Platform works and the best way is to get your hands dirty. Whether you wanna start as an administrator or a developer, I always recommend you face administrator skills at the beginning: a good developer should be a good administrator as well! As far as Marketer and Consultant paths are concerned, they are more related to your knowledge of specific products of the platform such as Marketing Cloud, Pardot, Field Service, Community Cloud, Einstein Analytics and many others. The Architect path brings you to the Mount Olympus of all certifications - the Technical Architect certification, which any Salesforce trailblazer aspire to get one day (and I’m one of them). Some think that owning a Salesforce certification doesn’t necessarily indicate your proficiency in the technologies involved but I do not agree with them. When I tried to get the Salesforce Advanced Administrator exam I really thought I had the required skills to pass but I failed...why? Because I didn’t study some of the topics and I wasn’t that skilled on such topics either (you’ll read this story in the book as well). That’s why I needed hours of study to pass the exam, and thanks to that deep study I learnt new Salesforce stuff and increased my proficiency in features I hadn’t actually ever used, making me the “most skilled” guy in my company regarding Omni-Channel or Salesforce Knowledge. This is an absolute win for both you and your company: certifications are meant to make you a trailblazer. Needless to say headhunters really love Salesforce certifications (my owning 20 certifications  attracts tens of contact requests on my social channels). Your book, Salesforce Advanced Administrator Certification Guide promises to give administrators a deeper knowledge of advanced Salesforce features for administrators. Why should one read this book? How is it different from other available Salesforce certification guides in the market? At first I want to say that the Salesforce Advanced Administrator Certification is a bit mistreated by administrators (as far as I’ve seen in my career): it is usually considered too hard or too complex for the skills you earn…”after all I’m already an administrator why should I become an advanced administrator”? You should my friend, the amount of things you learn is really huge, you’ll keep playing with features such as Lightning Knowledge, Omni-Channel, Live Chat, Lightning Content, features that maybe you’ve never used before, or exploring in depth the world of Salesforce automation with Process Builder, Lightning Flows, Entitlements and Approvals or knowing everything related to security and sharing of records (and many many more). Why should you choose this book? It covers extensively all required topics for the Salesforce Advanced Administrator certification keeping in mind the requirements for the exam as well. While the number of topics is too large for us to cover anything and everything for each topic, you’ll get a good amount of knowledge, suggestions and external references to ensure you reach the Salesforce Advanced Administrator certification goal. On the challenges faced by Salesforce administrators What are some of the challenges faced by Salesforce administrators today? How is Salesforce as a platform helping overcome these challenges? Can Salesforce administrators become developers too and vice versa? What is next for Salesforce? The biggest challenge that Salesforce admins face day after day is keeping pace with the extraordinarily growing Salesforce ecosystems: new companies join the Lightning Platform and new features are delivered release after release. It is more than mandatory that consultant companies and, in general, IT divisions reserve a percentage of their employees time for continuous learning, to allow Salesforce admins and devs to stay on track with the changing environment. Learning is a cost for sure, when you study you are not productive, but the benefits of a skilled and always on top employee overtakes its cost. And I see no obstacles for administrators to start their developer path as well: all they need is passion, curiosity and patience, Rome wasn’t built in a day and your developer skills won’t for sure. Trailhead is the starting point for any career path and I guess in the coming years we’ll see artificial intelligence absolutely stealing the show in Salesforce world and so admins should be prepared for the revolution that is taking place year after year. On making an impact in the Salesforce community You have created highly popular Salesforce browser extensions like ORGanizer. Tell us about how this came about? What does it take to build such successful products? Are you working on or planning to work on similar projects now? I said that boredom is my fuel: when I get bored I usually start a new project or a new hobby, and ORGanizer for Salesforce Chrome & Firefox extension (available at https://organizer.enree.co) is no different. It started as a personal project to ease my daily work with Salesforce projects, by adding little features that could speed up my administrative and coding tasks, while increasing my overall productivity. Then I thought, why not deliver this cool thing to my Salesforce Ohana? That’s where I believe the community took notice of me and it has remained one of the main reasons for my Salesforce MVP nomination. After the cool experience of writing a book, which is something that has been on my check list since I was a child, I have a few side projects related to Salesforce with some trailblazer friends, that I believe will have a great impact on the Ohana. And, why not, perhaps another book in 2020? Author Bio Enrico Murru is a Solution and Technical Architect at WebResults (an engineering company), an Italian platinum Salesforce partner, and an Independent Software Vendor (ISV). He has completed his MSc in Electronic Engineering at the University of Cagliari in 2007. In 2013, he launched a blog named Nerd @ Work. In 2016, he was nominated as the first Italian Salesforce MVP for his commitment to the Salesforce community. Then over the course of 3 years, Murru gained 20 Salesforce certifications, including the Salesforce Technical Architect certification. In 2016, he started one of the most popular projects, the ORGanizer for Salesforce Chrome and Firefox extension. You can follow him on Twitter @Enreeco, LinkedIn, GitHub, Trailblazer Community as well as on his personal blog page. Are you planning to embark on the journey of being a Salesforce Advanced Administrator? Confused about the various Salesforce certification programs and don’t know what to choose? Grab this book right now! The Salesforce Advanced Administrator Certification Guide will help you master data access security, monitoring and auditing, and understanding best practices for handling change management and data across organizations. What makes Salesforce Lightning Platform a powerful, fast and intuitive user interface What are the challenges of adopting AI-powered tools in Sales? How Salesforce can help Salesforce is buying Tableau in a $15.7 billion all-stock deal Salesforce’s open sourcing Centrifuge: A library for accelerating JVM restarts Build a custom Admin Home page in Salesforce CRM Lightning Experience
Read more
  • 0
  • 0
  • 26748

article-image-developers-dont-belong-on-a-pedestal-theyre-doing-a-job-like-everyone-else-april-wensel-on-toxic-tech-culture-and-compassionate-coding-interview
Richard Gall
02 Jul 2019
15 min read
Save for later

"Developers don't belong on a pedestal, they're doing a job like everyone else" - April Wensel on toxic tech culture and Compassionate Coding [Interview]

Richard Gall
02 Jul 2019
15 min read
It’s well known that there’s a toxic element to tech culture. And although it isn’t new, it has nevertheless surfaced and become more visible thanks to the increasing maturity of the platforms that are today shaping public discourse. As those platforms empower new voices to speak and allow new communities to organize, the very fabric of the culture on which many of them were built - hyper-masculine, competitive, and with scant disregard for the wider implications of their decisions on users - becomes the target of critique. But while everything from sexual harassment cover-ups to content moderation crises signal deep rooted issues inside the tech industry, substantially transforming tech’s cultural problems is a problem that’s more difficult to solve. It’s also one that many leading organizations and individuals seem to be unwilling to properly engage with. This is where April Wensel comes in. She’s made it her mission to help tackle issues of toxicity and ultimately transform tech culture with her organization, Compassionate Coding. What is Compassionate Coding? Compassionate Coding was launched in 2016 as a “response to a lot of the problems I saw in the tech industry with culture,” Wensel tells me when we spoke recently over Skype. “The common thread,” she explains, “was a lack of concern for human beings that are involved in technology or affected by technology.” This is particularly significant for Wensel. While it might be tempting to see the Google Walkout, the Cambridge Analytica scandal, and the controversy around Rekognition as nothing more than a collection of troubling but ultimately unrelated issues, it’s vital that we understand them together. [caption id="attachment_28750" align="alignright" width="300"] via compassionatecoding.com[/caption] “For things to really change - we can’t approach each issue as one problem,” Wensel says. “They really have the same root problem, which is this lack of compassion.” Compassion is an important and very deliberate word. It wasn’t chosen purely for its alliterative impact. “I chose compassion because I see compassion as a really rational thing; not just an intangible thing.” Compassion is, Wensel continues, “a more active form of empathy. Empathy allows you to feel what others are feeling, compassion allows you to see suffering, and - the important piece - to want to alleviate suffering.” Compassion as an antidote to toxic tech culture To talk about compassion in the tech industry is provocative. She tells me she recalls someone on Reddit describing the idea of compassionate coding as ‘girly '. But she tries to “tune out” online resistance, adopting a measured attitude: “whenever you have new, challenging ideas people get defensive.” Even if people aren’t aggressively opposed to her ideas, initially there was a distinct unwillingness to really engage with the ideas she was putting forward. “I… saw that it wasn’t cool to talk about these things. If you started talking about humans or whatnot, people are like oh, you must be a designer or you must be in product… No, I’m a developer. I just care about the people we’re impacting.” Crucial to this attitude is Wensel’s point that compassionate coding is something that can have real effects at every level. She describes it as “a new way of weighing decisions on a daily basis… it goes from high level things like what are we building? to low level things - what should I name this variable to make it easier for somebody in the future to understand?” Distributing power through diversity The context into which Compassionate Coding has entered the world is complex. High profile scandals need attention and action, but they are only the tip of the iceberg. They are symptomatic of low-lying problems that often pass unnoticed. Diversity is a good example of this. Although it’s often framed in the somewhat prosaic context of equal opportunities, it’s actually a powerful way of breaking apart privilege and the concentration of power that allows harmful products to be released and discrimination to find its way into organizational practices. By bringing people from a diverse range of backgrounds with different experiences into positions of authority and influence, the decisions that are made at all levels are supported by a greater awareness of context. In effect, decision making becomes more rigorous. Similarly, organizations themselves become safer and more welcoming places for employees from minority backgrounds because networks of support can form, making challenging malpractice or even abuse less of a risk professionally. This is something Wensel is well aware of. She takes umbrage with the concept of ‘diversity of thought’ which she sees as a way to mask a lack of genuine diversity. “A lot of companies claim they have diversity of thought…” she says, “that are all white men.” “You can’t really have true diversity of thought if everybody has come from the same background and hasn’t had any of the challenges that people from minority backgrounds might face.” The barriers to diversity are largely structural problems that can be felt far beyond tech. But according to Wensel, there are nevertheless cultural issues unique to the industry that are compounding the problem: “If you say you value diversity but really one of your values is the efficiency or perceived efficiency that comes when everyone thinks the same way then you have to realise that you’re gonna have to make some concessions in terms of creating a bit of discomfort when people are debating issues… because there is going to be some conflict when you create these diverse spaces.” Put another way, in an industry where you’re expected to move quickly and adapt, where you’re constantly looking for efficiency, diversity is always going to be an issue. It brings friction. For Wensel, the role Compassionate Coding can play in supporting diversity and inclusion is one where it helps to shift the industry mindset away from one that is scared of friction, to one where friction is vital if we’re to build better, safer, and more secure software. She points out that diversity isn’t just an initiative, it must be something that is constantly practiced: “Inclusion has to be a daily practice and so you need somebody who is in a position of power who can help establish inclusive practices,” she says. But it also needs to be something organizations need to invest in: “companies need to be paying people to do this because a lot of times the burden falls on underrepresented groups in the company and that’s not right.” Read next: Github Sponsors: Could corporate strategy eat FOSS culture for dinner? The problem with meritocracy If diversity can help unlock a better way of working in the tech industry, there are still other industry shibboleths that need to be slayed. According to Wensel, one of these is meritocracy. It is, she argues, often used as cover by those that are resistant to genuine diversity. “A lot of time in tech people want to talk about a meritocracy… [Recode co-founder] Kara Swisher says tech is more like a mirrortocracy because the people who succeed look like the ones who are already in the industry.” https://youtu.be/ng4sbQHCGLQ But what makes this problem worse is the fact that tech’s meritocracy is haunted by stereotypes and assumptions about what it means to be a developer. She points to a study done by IBM in the sixties that aimed to find out “what makes a good, strong programmer.” “They found among other things that programmers like puzzles, and they don’t like people… So it created a stereotype of what it means to be a good developer, and part of that was not liking people. And the reason that was so important - even though it was back in the sixties - is that IBM was a very influential company in terms of establishing tech culture,” Wensel says. Stack Overflow’s negative impact on tech culture What has further exacerbated this issue is how influential figures have helped to reinforce these stereotypes, effectively buying into the image of a programmer put forward in IBM’s research. In particular, Wensel calls out Stack Overflow and its founders Joel Spolsky and Jeff Atwood. “If you read through some of their old blogs from the early 2000s,” she says, “you can see a lot of the elements of the toxic culture that I talk about in so much of my work. Things like... hyper-competition… an over focus on aggressive competition… things like zero sum thinking. There’s an elitism - there’s not enough for everybody and some people are better than others.” Wensel suggests the attitudes of Atwood and Spolsky have been instrumental in forming the worst elements of the website “where the focus is not on helping people, but on accumulating points in the game of stack overflow.” Wensel detailed her experiences of Stack Overflow and offered an incisive critique of the website in a post on Medium in 2018. She reveals that although she has used Stack Overflow since its launch in 2008 (the year she graduated from her Computer Science class) “the condescending and blatantly rude responses on the site” have dissuaded her from ever actually creating an account. Although the Compassionate Code founder can see that the site is trying to change things, she believes it can still do a lot more (in her post she adds this response from Stack overflow employee Joe Friend). The problem, however, is that this would be a risk for the company. “They really have to be willing to alienate their audience - the ones who are contributing to the toxic culture.” Ultimately this highlights the problem facing many companies and communities in the tech industry - inclusivity and diversity aren’t things that can simply be integrated into established patterns and beliefs. Those beliefs and values need to change too. Which can, of course be painful. Dismantling the hierarchy of tech skills Again, it’s important to note that Wensel’s criticisms aren’t just on the grounds of civility or accessibility. It’s ultimately bad for the industry as a whole and bad for users. It helps to cultivate an engineering culture where certain skills are overvalued while others are excluded. This has consequences for how we view ourselves in the industry (we're never good enough, and we constantly have to compete), but it also means the sort of work and thought that should go into building and delivering software is viewed as less important. “None of this is productive and none of this is creating value. We need people doing all of these roles, and so which one of these has more prestige shouldn’t be an issue” Wensel argues. “That’s why one of very clear indications that there’s a problem in the culture is the fact that we are obsessed with the need to rank skills... software projects are failing for people reasons. And yet people who are good with people and technology are seen as too soft… they’re put in a box of not being technical.” Wensel argues that we need to stop worrying about who is and who isn’t a developer. “There’s no such thing as a real developer. If you write code you’re a developer... that’s enough… Developers are no better than designers, or product managers, or salespeople… that hierarchy is even more entrenched because it’s often reflected in salaries - so developers get paid disproportionately more than all these other roles.” The myth of scarcity and the tech skills gap What’s more, Wensel believes this hierarchy of programming skills is actually helping to perpetuate the notion of a tech skills gap. She believes the idea that there is a scarcity of “tech talent” is a “myth.” “I think there’s tons of talent in tech that’s being overlooked for reasons of unconscious bias, stereotypes…” she explains. “Once we start to bring in these people to the table who are out there already - very talented, very skilled - it will start to melt away this whole putting developers on a pedestal… developers don’t belong on a pedestal, they’re just doing a job like anybody else.” Wensel believes we will - and need to - move towards a world where programming skills lose their “prestige”. Having Python or React on your CV, for example, should really be no different to saying you know how to use Excel. “As these skills become seen for what they are, which is just something that anybody can learn if they put in the time, then I think that the prestige around them will be reduced.” How Agile is changing what it means to be a developer We’re moving towards a world where the solipsism of the valorization of technical skill becomes outdated thanks to broader industry trends. With DevOps forcing developers to become accountable for the full lifecycle of their code, and distributed systems engineering requiring a holistic awareness of a complex network of dependencies, it’s clear that more sensitivity about how your code is interacting with and impacting users in the real world is more important in software engineering than it ever has. “Over and over again I see both in formal studies and anecdotally… what’s causing software projects to fail or to be delayed... are people problems. Coordination problems, planning problems resourcing, all of that - not purely technical problems,” says Wensel. That said, Wensel nevertheless views Agile as a trend that’s positive for the industry. “A lot of the ideas behind agile software development are really positive in a lot of ways I see it as the first step in bringing emotional intelligence to the software team because you’re asked to consider the end user…” Read next: DevOps Engineering and Full-Stack Development – 2 Sides of the Same Agile Coin However, she also says that software engineering practices and philosophies like Agile only go so far. “The problem is that they [proponents of Agile] didn’t bring in the ethics there. So you can still create a lot of value very efficiently with agile development without considering the long term impact.” Agile is a good context for Wensel to drive her mission forward - but it can’t improve things on it own. Read next: Honeycomb CEO Charity Majors discusses observability and dealing with “the coming armageddon of complexity” [Interview] Putting Compassionate Coding into practice It’s clear that Compassionate Coding is needed in today’s software industry. Yes, tech culture’s toxicity is damaging and dangerous for everyone, but it’s also not fit for purpose. It’s stopping us from evolving and building the software people actually need. Think of it this way: it’s stopping us from putting users first at a time when the very idea of the individual feels vulnerable, thanks to a whirlwind of reactionary politics and rampant, unsustainable capitalism. However, it’s important that we actually see Compassionate Coding as something that can be practiced, both by individuals and organizations. The 4 levels of compassionate coding Wensel explained compassionate coding as involving 4 key ‘levels’. These levels turn the concept into something practical, that every individual and team can actually go and do themselves. “It’s how you treat yourself with compassion, how you treat your coworkers, your collaborators with compassion, how you treat your direct users of the software you’re creating… and how you treat the community at large who may or may not be people who use your product,” she says. Wensel is not only continuing to deliver training sessions and keynotes for her clients, but is also writing a book which will make her ideas more accessible. I asked her what advice she would offer individuals and businesses that want to follow her lead now. “The biggest thing people can do,” she says, “is to analyze their own thinking… Do a bit of meta-cognition to understand how do I think? Where do I have biases? At an organizational level, businesses should be “prioritizing talking about these issues, making it safe to talk about these issues, hiring people who understand these issues and can improve your company in these ways” she says. The importance of the individual in tackling tech's toxicity But Wensel still believes in the importance of individuals in enacting change. “It’s humans all the way down and all the way up… Leadership in a company and [the issue of] who makes decisions is just... another set of humans, and so I think changing individuals is really powerful.” Her approach is ultimately one that espouses the values of Compassionate Coding. “You can’t control the outcome but you can control the actions you take. So I have a lot of faith in the change that motivated individuals can make.” If everyone in the industry could adopt that attitude we’d surely be some way towards not better professional lives and better experiences and products for users. Follow April on Twitter: @aprilwensel  Other projects that are making the tech industry better April cited a number of organizations that she believes are doing great and important work across the tech industry: Project Include, an organization that wants to accelerate diversity in the industry. Black Girls Code, which aims to improve the number of women of color in the digital sector. Elephant in the Valley, which is tackling gender disparity in Silicon Valley. Kapor Center, removing barriers for underrepresented groups in tech. Learn more about the issues they're helping to solve, and support them if you can.
Read more
  • 0
  • 0
  • 26187

article-image-everybody-can-benefit-odoo-development-an-interview-yenthe-van-ginneken
Sugandha Lahoti
29 Jun 2018
9 min read
Save for later

“Everybody can benefit from adopting Odoo, whether you’re a small start-up or a giant tech company” - An interview with Odoo community hero, Yenthe Van Ginneken

Sugandha Lahoti
29 Jun 2018
9 min read
Odoo is one of the fastest growing open source, business application development software products available. It comes with: Powerful GUI, Performance optimization, Integrated in-app purchase features Fast-growing community to transform and modernize businesses We recently interviewed Yenthe Van Ginneken, an Odoo developer, highly active in the Odoo community and recipient of Odoo best contributor of the year 2016 and Odoo community hero 2017. He spoke to us about his journey with Odoo, his thoughts on Odoo’s past, present and future, and the Odoo community. Expert's Bio Yenthe Van Ginneken, currently the technical team leader at Odoo Experts, has been an Odoo developer for over four years. He has won two awards, “Best contributor of the year 2016” and the “Odoo community hero” award in 2017. He loves improving software and teaching other people the best practices for Odoo development on his blog. You can read his Odoo blog, follow him on Twitter or reach out to him on LinkedIn. Key Takeaways Odoo is scalable and flexible to the extent that everyone, from a small startup to a giant tech company can benefit from it. It is ahead of quite a lot of ERP systems with its clean UI, advanced modules integration and the flexibility of its technical framework. Python is the preferred language of choice among most developers that want to use the Odoo framework, especially for automating and scaling tasks. The Odoo community is diverse and vast. By contributing and regularly interacting with other members, you will gain deeper insights into many different aspects of Odoo development. A great way to learn to develop in Odoo and quickly grow is actually by helping in the community. Odoo 12 will reportedly improve data processing, better report insights, and support for OCR (Optical Character Recognition) for handling documents among other exciting updates. Full Interview On who should use Odoo Odoo is more than an ERP tool. According to you, What is Odoo? Who will benefit from adopting Odoo? What made you choose Odoo?   For me, Odoo is more than an ERP. Odoo literally allows me to make any module or functionality that I can think of. Since Odoo is so flexible and scalable I believe that almost everybody can benefit from adopting Odoo. Whether you’re a small start-up or a giant tech company. The most important part to be able to benefit from adopting Odoo is adjusting the processes and mindset to use Odoo, not adjusting Odoo for the company. The projects that work the best and have the best benefit are those that don’t over-engineer and try to focus on the main company processes. I personally chose Odoo after I got an opportunity to become an Odoo developer at a company in Belgium. After the job offer I visited Odoo.com and saw the massive amount of functionalities in Odoo (while being free!) and I was genuinely amazed. After looking at the technical framework and all the default options provided by the framework I was sure that I would love to develop and implement in Odoo. Since that day I never stopped working with Odoo. On journey from OpenERP to Odoo Odoo started off as OpenERP and then in 2014, it moved beyond just ERP and was renamed Odoo. How has Odoo’s journey been so far since then? What do you think are the key milestones achieved by Odoo till date? Since the renaming from OpenERP to Odoo the company has seen a rapid growth. A bit after changing the name Odoo also introduced the enterprise version which was, in my opinion, the turning point for Odoo S.A. It allowed Odoo to keep its open source strength and market share while also gathering funds to fund the ongoing growth of the product. The big investments that are being made in the Research and Development team allow them to keep improving year after year. The main strengths and key milestones from Odoo are absolutely its flexibility, a great framework and the fact that most of the possibilities are already in Odoo by default. On the drive behind contributing to the Odoo community You are highly active in the Odoo community. How did you get into contributing for Odoo? How has this experience improved you as a developer? According to you, what are the key challenges the Odoo community is facing currently? My very first contribution started in the second half of 2014 and weren’t very significant at first. I noticed that Odoo 8, at that point the newest version, was not very well translated and had a lot of inconsistency so I started translating it in Dutch. From there on I noticed that it could have had quite a big impact and in fact could improve the ERP. It didn’t take long before I started contributing in other ways. Reporting issues, fixing bugs, maintaining bug reports and helping other people on the official help forums. By contributing to all these different subjects I got introduced to more domains and gained more insights. Thanks to my involvement with the community, I’ve learned that there is more than one side to developing and implementing projects. I believe it made me a better programmer and made me think a lot more about ways to code custom development for projects. Without being active in a community and contributing you’ll be blindsided by your own perspective. It is a great way to get challenged and you’ll see more cases by being active in the community than you could ever see on your own. The Odoo community faces a few challenges at this point. It is difficult to maintain the right balance between the enterprise version and community (free) version. There are not a lot of very active contributors to the official Odoo code and Odoo is behind on handling fixes/bug reports made by community members. This results in some community members not feeling appreciated or heard. Hiring a second community manager might be a good way to resolve these issues though. The most difficult challenge for both Odoo and the Odoo community is to make everybody feel heard and give every person the ability to contribute in the way he or she can. When there is enough help from Odoo and the community feels supported there is a possibility for a great and thriving community. On how to learn Odoo effectively As a person who has a strong hold over Odoo development, what is the typical learning curve for someone getting into Odoo, as a consultant? What is the best way to start developing in Odoo? What should one watch out for while learning? The learning curve can be quite long and can have its challenges. Usually, if you don’t have any experience with Odoo and only know basic Python it’ll take about six months before you really get to know the ins and outs of Odoo. The best way to learn to develop Odoo is probably the same as with most things in technology: dive in! Make sure you get the basics right and understand how the main functionalities work before going deeper. A great way to learn to develop in Odoo and to quickly grow is actually by helping in the community. You can get insight and help from experienced developers while also contributing to the community, it’s a win-win. Start small and build your way up to the details. It is important to find good documentation and tutorials though. At the moment there are still quite some blog posts and tutorials that are from quite a low quality. Because of this I actually started writing my own tutorials, which explain concepts step by step with samples. You can find it at https://odoo.yenthevg.com Editor’s note: Check out our collection of Odoo Books and Videos to master Odoo development. On the upcoming Odoo 12 release Odoo 12 is expected to be released later this year. What’s got you excited about this new release? Quite a lot! Every release has loads of new features that are announced and it’s an exciting time, every time. The introduction of a report designer for functional people is one of the best (known) new features. The improved reporting tools for data insight will become a great improvement too. The biggest announcements are made at Odoo Experience in October and are not publicly available yet so we’ll have to wait for that. On the future of ERP There is a lot happening in the area of ERP and BI: self-service analytics, real-time analytics, agile BI development etc. Where do you foresee the ERP market headed? We've seen ERP/CRM systems getting powerful inbuilt analytics systems, what do you think is next for the industry? What is Odoo’s role here? As with any sector in IT, a lot is becoming very data-driven. In the future integration and usage of data will only grow. I expect the combination of BI and AI to become a powerful way to process and handle data on unseen scales. Odoo itself has already hinted at improved data processing, better report insights and support for OCR (Optical Character Recognition) for handling documents. Odoo has been ahead of quite a lot of ERP systems with its clean UI, advanced modules integration and the flexibility of its technical framework for years. I expect Odoo will also be leading the way for handling all this data and getting important statistics out of it. I’m quite sure it is only a matter of time before Odoo starts working on even better BI reporting and tools. On Python and automation Automation is everywhere today and becoming an integral part of organizations and processes. Python and automation have gone hand in hand since Python’s early days. Today Python is one of the top programming languages. How do you see Python’s evolution over the years in the area of automation? What are the top ways you use Python for automation, today? It is for a reason that Python is so popular. It is flexible, quite quick to program with and the options are virtually endless. In the next years, Python will only become more popular and this will also be the case for automation projects made with Python. I personally use the Odoo framework with Python as a backbone for nearly everything that I automate (and in fact also for non-automated tasks). The projects vary from automatically handling stock moves to automatically updating remote instances to automatically getting full diagnostic reports. The combination of the programming language and the framework from Odoo allows me to automate tasks and deploy them on a big scale. ERP tool in focus: Odoo 11 How to Scaffold a New module in Odoo 11 A step by step guide to creating Odoo Addon Modules
Read more
  • 0
  • 0
  • 26033

article-image-unlocking-insights-how-power-bi-empowers-analytics-for-all-users
Gogula Aryalingam
29 Nov 2024
5 min read
Save for later

Unlocking Insights: How Power BI Empowers Analytics for All Users

Gogula Aryalingam
29 Nov 2024
5 min read
IntroductionIn today’s data-driven world, businesses rely heavily on robust tools to transform raw data into actionable insights. Among these tools, Microsoft Power BI stands out as a leader, renowned for its versatility and user-friendliness. From its humble beginnings as an Excel add-in, Power BI has evolved into a comprehensive enterprise business intelligence platform, competing with industry giants like Tableau and Qlik. This journey of transformation reflects not only Microsoft’s innovation but also the growing need for accessible, scalable analytics solutions.As a data specialist who has transitioned from traditional data warehousing to modern analytics platforms, I’ve witnessed firsthand how Power BI empowers both technical and non-technical users. It has become an indispensable tool, offering capabilities that bridge the gap between data modeling and visualization, catering to everyone from citizen developers to seasoned data analysts. This article explores the evolution of Power BI, its role in democratizing data analytics, and its integration into broader solutions like Microsoft Fabric, highlighting why mastering Power BI is critical for anyone pursuing a career in analytics.The Changing Tide for Data Analysts When you think of business intelligence in the modern era, Power BI is often the first tool that comes to mind. However, this wasn't always the case. Originally launched as an add-in for Microsoft Excel, Power BI quickly evolved into a comprehensive enterprise business intelligence platform in a few years competing with the likes of Qlik and Tableau—a true testament to its capabilities. As a data specialist, what really impresses me about Power BI's evolution is how Microsoft has continuously improved its user-friendliness, making both data modeling and visualizing more accessible, catering to both technical professionals and business users.  As a data specialist, initially working with traditional data warehousing, and now with modern data lakehouse-based analytics platforms, I’ve come to appreciate the capabilities that Power BI brings to the table. It empowers me to go beyond the basics, allowing me to develop detailed semantic layers and create impactful visualizations that turn raw data into actionable insights. This capability is crucial in delivering truly comprehensive, end-to-end analytics solutions. For technical folk like me, by building on our experiences working with these architectures and the deep understanding of the technologies and concepts that drive them, integrating Power BI into the workflow is a smooth and intuitive process. The transition to including Power BI in my solutions feels almost like a natural progression, as it seamlessly complements and enhances the existing frameworks I work with. It's become an indispensable tool in my data toolkit, helping me to push the boundaries of what's possible in analytics. In recent years, there has been a noticeable increase in the number of citizen developers and citizen data scientists. These are non-technical professionals who are well-versed in their business domains and dabble with technology to create their own solutions. This trend has driven the development of a range of low-code/no-code, visual tools such as Coda, Appian, OutSystems, Shopify, and Microsoft’s Power Platform. At the same time, the role of the data analyst has significantly expanded. More organizations are now entrusting data analysts with responsibilities that were traditionally handled by technology or IT departments. These include tasks like reporting, generating insights, data governance, and even managing the organization’s entire analytics function. This shift reflects the growing importance of data analytics in driving business decisions and operations. As a data specialist, I’ve been particularly impressed by how Power BI has evolved in terms of user-friendliness, catering not just to tech-savvy professionals but also to business users. Microsoft has continuously refined Power BI, simplifying complex tasks and making it easy for users of all skill levels to connect, model, and visualize data. This focus on usability is what makes Power BI such a powerful tool, accessible to a wide range of users. For non-technical users, Power BI offers a short learning curve, enabling them to connect to and model data for reporting without needing to rely on Excel, which they might be more familiar with. Once the data is modeled, they can explore a variety of visualization options to derive insights. Moreover, Power BI’s capabilities extend beyond simple reporting, allowing users to scale their work into a full-fledged enterprise business intelligence system. Many data analysts are now looking to deepen their understanding of the broader solutions and technologies that support their work. This is where Microsoft Fabric becomes essential. Fabric extends Power BI by transforming it into a comprehensive, end-to-end analytics platform, incorporating data lakes, data warehouses, data marts, data engineering, data science, and more. With these advanced capabilities, technical work becomes significantly easier, enabling data analysts to take their skills to the next level and realize their full potential in driving analytics solutions.  If you're considering a career in analytics and business intelligence, it's crucial to master the fundamentals and gain a comprehensive understanding of the necessary skills. With the field rapidly evolving, staying ahead means equipping yourself with the right knowledge to confidently join this dynamic industry. The Complete Power BI Interview Guide is designed to guide you through this process, providing the essential insights and tools you need to jump on board and thrive in your analytics journey. ConclusionConclusionMicrosoft Power BI has redefined the analytics landscape by making advanced business intelligence capabilities accessible to a wide audience, from technical professionals to business users. Its seamless integration into modern analytics workflows and its ability to support end-to-end solutions make it an invaluable tool in today’s data-centric environment. With the rise of citizen developers and expanded responsibilities for data analysts, tools like Power BI and platforms like Microsoft Fabric are paving the way for more innovative and comprehensive analytics solutions.For aspiring professionals, understanding the fundamentals of Power BI and its ecosystem is key to thriving in the analytics field. If you're looking to master Power BI and gain the confidence to excel in interviews and real-world scenarios, The Complete Power BI Interview Guide is an invaluable resource. From the core PowerBI concepts to interview preparation and onboarding tips and tricks, The Complete Power BI Interview Guide is the ultimate resource for beginners and aspiring Power BI job seekers who want to stand out from the competition.Author BioGogula is an analytics and BI architect born and raised in Sri Lanka. His childhood was spent dreaming, while most of his adulthood was and is spent working with technology. He currently works for a technology and services company based out of Colombo. He has accumulated close to 20 years of experience working with a diverse range of customers across various domains, including insurance, healthcare, logistics, manufacturing, fashion, F&B, K-12, and tertiary education. Throughout his career, he has undertaken multiple roles, including managing delivery, architecting, designing, and developing data & AI solutions. Gogula is a recipient of the Microsoft MVP award more than 15 times, has contributed to the development and standardization of Microsoft certifications, and holds over 15 data & AI certifications. In his leisure time, he enjoys experimenting with and writing about technology, as well as organizing and speaking at technology meetups. 
Read more
  • 0
  • 0
  • 25827

article-image-with-python-you-can-create-self-explanatory-concise-and-engaging-data-visuals-and-present-insights-that-impact-your-business-tim-grosmann-and-mario-dobler-interview
Savia Lobo
10 Nov 2018
6 min read
Save for later

“With Python, you can create self-explanatory, concise, and engaging data visuals, and present insights that impact your business” - Tim Großmann and Mario Döbler [Interview]

Savia Lobo
10 Nov 2018
6 min read
Data today is the world’s most important resource. However, without properly visualizing your data to discover meaningful insights, it’s useless. Creating visualizations helps in getting a clearer and concise view of the data, making it more tangible for (non-technical) audiences. To further illustrate this, below are questions aimed at giving you an idea why data visualization is so important and why Python should be your choice. In a recent interview, Tim Großmann and Mario Döbler, the authors of the course titled, ‘Data Visualization with Python’, shared with us the importance of Data visualization and why Python is the best fit to carry out Data Visualization. Key Takeaways Data visualization is a great way, and sometimes the only way, to make sense of the constantly growing mountain of data being generated today. With Python, you can create self-explanatory, concise, and engaging data visuals, and present insights that impact your business. Your data visualizations will make information more tangible for the stakeholders while telling them an interesting story. Visualizations are a great tool to transfer your understanding of the data to a less technical co-worker. This builds a faster and better understanding of data. Python is the most requested and used language in the industry. Its ease of use and the speed at which you can manipulate and visualize data, combined with the number of available libraries makes Python the best choice. Full Interview Why is Data Visualization important? What problem is it solving? As the amount of data grows, the need for developers with knowledge of data analytics and especially data visualization spikes. In recent years we have experienced an exponential growth of data. Currently, the amount of data doubles every two years. For example, more than eight thousand tweets are sent per second; and more than eight hundred photos are uploaded to Instagram per second. To cope with the large amounts of data, visualization is essential to make it more accessible and understandable. Everyone has heard of the saying that a picture is worth a thousand words. Humans process visual data better and faster than any other type of data. Another important point is that data is not necessarily the same as information. Often people aren’t interested in the data, but in some information hidden in the data. Data visualization is a great tool to discover the hidden patterns and reveal the relevant information. It bridges the gap between quantitative data and human reasoning, or in other words, visualization turns data into meaningful information. What other similar solutions or tools are out there? Why is Python better? Data visualizations can be created in many ways using many different tools. MATLAB and R are two of the available languages that are heavily used in the field of data science and data visualization. There are also some non-coding tools like Tableau which are used to quickly create some basic visualizations. However, Python is the most requested and used language in the industry. Its ease of use and the speed at which you can manipulate and visualize data, combined with the number of available libraries makes Python the best choice. In addition to all the mentioned perks, Python has an incredibly big ecosystem with thousands of active developers. Python really differs in a way that allows users to also build their own small additions to the tools they use, if necessary. There are examples of pretty much everything online for you to use, modify, and learn from. How can Data Visualization help developers? Give specific examples of how it can solve a problem. Working with, and especially understanding, large amounts of data can be a hard task. Without visualizations, this might even be impossible for some datasets. Especially if you need to transfer your understanding of the data to a less technical co-worker, visualizations are a great tool for a faster and better understanding of data. In general, looking at your data visualized often speaks more than a thousand words. Imagine getting a dataset which only consists of numerical columns. Getting some good insights into this data without visualizations is impossible. However, even with some simple plots, you can often improve your understanding of even the most difficult datasets. Think back to the last time you had to give a presentation about your findings and all you had was a table with numerical values in it. You understood it, but your colleagues sat there and scratched their heads. Instead had you created some simple visualizations, you would have impressed the entire team with your results. What are some best practices for learning/using Data Visualization with Python? Some of the best practices you should keep in mind while visualizing data with Python are: Start looking and experimenting with examples Start from scratch and build on it Make full use of documentation Use every opportunity you have with data to visualize it To know more about the best practices in detail, read our detailed notes on 4 tips for learning Data Visualization with Python. What are some myths/misconceptions surrounding Data Visualization with Python? Data visualizations are just for data scientists Its technologies are difficult to learn Data visualization isn’t needed for data insights Data visualization takes a lot of time Read about these myths in detail in our article, ‘Python Data Visualization myths you should know about’. Data visualization in combination with Python is an essential skill when working with data. When properly utilized, it is a powerful combination that not only enables you to get better insights into your data but also gives you the tool to communicate results better. Data nowadays is everywhere so developers of every discipline should be able to work with it and understand it. About the authors Tim Großmann Tim Großmann is a CS student with interest in diverse topics ranging from AI to IoT. He previously worked at the Bosch Center for Artificial Intelligence in Silicon Valley in the field of big data engineering. He’s highly involved in different Open Source projects and actively speaks at meetups and conferences about his projects and experiences. Mario Döbler Mario Döbler is a graduate student with a focus in deep learning and AI. He previously worked at the Bosch Center for Artificial Intelligence in Silicon Valley in the field of deep learning, using state-of-the-art algorithms to develop cutting-edge products. Currently, he dedicates himself to apply deep learning to medical data to make health care accessible to everyone. Setting up Apache Druid in Hadoop for Data visualizations [Tutorial] 8 ways to improve your data visualizations Getting started with Data Visualization in Tableau  
Read more
  • 0
  • 0
  • 25646
Unlock access to the largest independent learning library in Tech for FREE!
Get unlimited access to 7500+ expert-authored eBooks and video courses covering every tech area you can think of.
Renews at €18.99/month. Cancel anytime
article-image-how-two-junior-intuit-engineers-helped-their-team-adopt-kotlin-within-a-month
Richard Gall
25 Nov 2019
6 min read
Save for later

How two junior Intuit engineers helped their team adopt Kotlin within a month

Richard Gall
25 Nov 2019
6 min read
Change might feel like a natural part of working in the software industry. But in truth it's not natural at all; it takes a hell of a lot of effort to do things differently. That's what software developers Shelby Cohen and Katie Levy found out when they decided that Kotlin could be a better programming language option when it came to their company's engineering team. As two relatively junior developers at financial software provider Intuit, Shelby and Katie didn't only have to take on the challenge of building out training programs and providing resources to thousands of developers across Intuit, they also had to negotiate internal hierarchies and politics that can prove resistant to change. To learn more about what this process was like, as well as why they're so passionate about Kotlin, I spoke to them over email. Visit the Packt store to explore Kotlin eBooks and videos. How did you get started in software engineering? Shelby Cohen: I’ve always been interested in solving problems and engaging with new challenges. In high school I really enjoyed math and one of my math teachers encouraged me to join the High School Robotics Club. It was all men and I didn’t feel like I fit in. With the help and encouragement of my teacher I helped start the Women Robotics Club at my high school. This is where I was exposed to programming for the first time and that inspired me to study computer science in college. During my junior year, I learned about Intuit’s co-op program at a hackathon and thought it was a great opportunity. I then travelled from New York to San Diego to spend a semester working at Intuit. I learned so much and was exposed to a lot of mentorship opportunities which led to me accepting a full time position as a Software Engineer I in 2017. Intuit really values teaching their employees and helping them continue to grow and develop so it felt like a natural fit. Why Kotlin? What’s unique about Kotlin? Katie Levy: Kotlin is an open source, cross-platform programming language, designed to interoperate with Java. What’s nice about it is that it allows for an easy transition to start using a functional programming style while also being safer and more concise than Java code. Kotlin is unique because it is very clean, simple, clear and removes a lot of the redundant, boilerplate code that’s in Java which allows the developer to focus on the business logic. It’s my favorite language to program in as I can write high quality code faster by using its built-in language features. It can be used on any application running on the JVM, including Spring boot backend services, Android apps, and even JavaScript applications. How/where did you learn about it? Was it an immediate thing or did it take time for you to decide to do this? Shelby: I volunteered at KotlinConf back in 2018, and it was such a great opportunity to meet a lot of the engineers and staff from Jetbrains. I got to meet a senior executive at Jetbrains and shared some of the projects I was working on at Intuit. He asked to be a guest on his podcast, TalkingKotlin, and through this conference I got to know some of the most influential people in the Kotlin community. How did people respond? Were they resistant to something new? Katie: My team was definitely hesitant to start learning the language. One of the engineers on my team was especially resistant and tried to identify flaws in the language, using anything he could come up with as a reason why we shouldn’t use Kotlin. In those cases it’s important to identify the real issue the engineers are having with the new language — is it the language itself or is it something else? With that particular engineer, I found out that he was feeling like he didn’t have the bandwidth to take on the work he was being assigned. To combat this, I created a training program for the team so we could learn together, build up the team’s domain knowledge on the language, and so everyone’s workload was more visible. How did you go about driving adoption? Katie: Influencing and driving change is a hard project to scope. It can mean many different things to different people. For us, when we were starting out, we wanted to introduce 500 engineers to Kotlin, and wanted 90% of them to start coding in Kotlin. We ended up exceeding our goal, reaching 370 engineers internal to Intuit and 4,329 external to Intuit. We want to improve the industry by encouraging engineers to develop in a more concise and less error-prone language. More recently, we were able to present on Kotlin to over 500 software engineers at LambdaWorld in Spain. Afterward, we had engineers wanting to take pictures with us and all saying they want to start using the language. We found that speaking at conferences helped us meet our goals, and scale our efforts. What were the challenges? Shelby: In addition to some of the initial resistance, one of the biggest challenges is the internal hierarchy. When I introduced Kotlin to my team, a lot of engineers were more senior than me and at the time, I wasn’t confident about the value that I was bringing to the team. I implemented group code reviews, sent out resources, and walked the team through examples as they were learning Kotlin. Once the team had a good foundation for Kotlin, I implemented a flatter teaching structure and encouraged everyone to learn and teach each other a specific part of the language. This was really effective because everyone learns from different teaching styles and team members felt more empowered in their work. Have you learned anything else about Kotlin throughout the process? And about engineering in general? Shelby: One of the most valuable things I learned from this experience is don’t be afraid to ask for advice on how to influence at scale and connect with others who have driven change at your company. This way you can learn from other’s experiences. Katie and I reached out to a lot of leaders making an impact in their area of expertise to learn from them and get feedback and advice on our journey. This is something we will continue to do as we keep learning and facing other challenging problems. Thanks to Shelby and Katie for talking to us - it's clear they have a lot of passion for Kotlin, but more importantly they also have a great sense of how to engage and support other developers. Follow Shelby on Twitter: @shelbyc0hen Follow Katie on Twitter: @klevy110
Read more
  • 0
  • 0
  • 25500

article-image-functional-python-programming-interview-steven-lott
Aaron Lazar
17 May 2018
8 min read
Save for later

Why functional programming in Python matters: Interview with best selling author, Steven Lott

Aaron Lazar
17 May 2018
8 min read
Python is currently one of the most popular and desired programming languages. Primarily for its simplicity, agility and ability to be used across a variety of software development projects. Although an Object Oriented language, Python supports an array of programming paradigms, including Functional Programming. Functional Programming is a paradigm that treats computation as the evaluation of math functions. It’s quite advantageous as it allows for efficient parallel programming and error-free code. We recently interviewed Steven Lott, a true Python professional and best-selling author of a number of Python books. Steven talks a bit about modern Python and how the language adapts well to the Functional paradigm, offering developers a range of solutions to build modern, cloud based applications. He also talks about his most recent book, the second edition of his best seller, Functional Python Programming, and how it will benefit developers looking to enter the world of functional programming with Python. About Steven Steven F. Lott has been programming since the 70s, when computers were large, expensive, and rare. As a contract software developer and architect, he has worked on hundreds of projects, from very small to very large. He's been using Python to solve business problems for over 10 years. Steven is a technomad who lives in various places on the east coast of the U.S. Follow his technology blog to stay updated on the latest trends in tech. You may also connect with him on LinkedIn. Key Takeaways Why learn Python? One of the major reasons developers appreciate Python, is because it’s simple, and has the ability to create succinct and expressive programs. For example, data scientists prefer Python because they can build sophisticated analytical tools using simple functions and produce useful results. Why Functional programming? The functional programming paradigm forms the perfect foundation for developers and architects to build and design modern architectures like Serverless. But Python isn’t inherently functional. Although an object oriented language at heart, Python can create higher order functions and other functional features. Python 3 has made this easier. Steven’s Python 4 wish list: In future versions of Python, Steven hopes to see a wider use of Unicode operator characters like × in addition, * for multiplication, and ÷ in addition to / for division. Also, he expects PyPy and RPython projects to be more widely used; future Pythons versions will benefit from optimizations and restructuring the interpreter. One of the most helpful features of Python 3 are type hints, which allow one to write clear and implicitly documented code while preventing the invoking of methods with wrong data types. Steven’s latest edition of Functional programming with Python, explores type hints in depth along with core language features such as lambdas, generator expressions, functions, and callable objects. Full Interview Python is one of the top programming languages. List down top 3 features of Python that make programmers love it.. It seems like programmers love Python primarily because it allows them to create succinct, expressive programs. Before long, they learn the vast library of code - is another reason for Python's immense popularity. Many people adopt Python because of the low barrier to entry: it really is as simple as download and start working. You've been working with Python for over a decade. How has your experience been with Python as a primary development language? Over my 40-year career, I've used a variety of languages. And I've found Python to be extremely productive. A team can build and deploy microservices-based applications at a tremendous pace. Data scientists can build sophisticated analytical tools using simple functions to produce useful results without the overheads of complex compile and build environments. Back when Python 3 just came into existence, we saw certain resistance to the notion of Python being apt for functional programming. How would you say Python has progressed since then? At its core, Python is an object-oriented language. Consequently some functional features aren't central. One of the essential functional design tools -- creating higher-order functions -- has always been part of Python. The wider use of generator functions in Python 3 has made functional Python programming much more common. Many Python applications are hybrids, mixing object-oriented and functional features of the language. Now that type hints are available, it becomes practical to use mypy to confirm that the code is very likely to work properly. For a developer who's picking up the Functional Programming paradigm for the first time, what do you think are the prerequisites? Functional programming is closely aligned to the core mathematical ideas of functions and functional composition. As a consequence, a minimal background in programming could be advantageous to help leverage essential function definitions and avoid needless state change. For programmers already heavily invested in procedural programming, it may be helpful to set the idea of stateful objects aside. In the above context, how does your book, Functional Python Programming, Second Edition, prepare its readers to be industry ready? What are the key takeaways for readers from your title and how does it help with the learning curve? The examples in the book are related to exploratory data analysis, an important skill in the broader area of data science. They also focus on the standard library, allowing someone to apply the functional design approach to other libraries and tools. I think a focus on the core language features (e.g., lambdas, generator expressions, functions, callable objects) provide a foundation that allows a programmer to apply the core ideas more widely to different kinds of problems and other software packages. What new and updated content is available in this edition, for developers who've purchased your previous book? Almost all of the examples have been rewritten to include type hints. This can be an important quality check helping to ensure the Python code works. When used with doctest examples, it becomes relatively easy to provide reliable, correct code. In a few cases, external packages (i.e., the pymonad library) don't have type hints and the examples reflect this gap. Can you throw some light on Functional Reactive Programming and how FRP with Python is boosting the implementation of modern architectures like Cloud Native and Serverless? The central idea of serverless programming -- a collection of isolated functions -- fits the functional programming paradigm very elegantly. The processing is generally stateless, with stand-alone functions waiting for their inputs. Ideas like "choreography" of web services work with this idea of stateless functions that respond to an input by producing an output. This leads to careful separation of persistence and state change from the other transformational processing. This helps create software with easy-to-understand behavior and implementation code that's very expressive of the algorithm. As Python inches towards a 4.0, what do you think should/can be changed/rectified in the language, for the better? At some point, I expect the PyPy and RPython projects to create some optimizations leading to a fundamental restructuring of the interpreter. Perhaps these changes could remove the need for the GIL (Global Interpreter Lock) by exposing a minimal kernel of code that requires exclusive access to internal data structures. Of more general interest, I'd hope to see wider use of Unicode operator characters like × in addition to * for multiplication, and ÷ in addition to / for division. Perhaps ∻ could be adopted for truncated division. This could also lead to use of ℝ instead of float and ℤ instead of int providing a more mathematical look to type hints. It would be nice to expand the use of the Unicode character set to create more readable programs. List down 3 reasons for developers to choose your book as the book of choice for Functional programming in Python. Programmers who want to create succinct and expressive code often find functional design to help them fulfill this. The Functional Python Programming book provides extensive examples that show multiple ways to achieve this goal. In many cases, generator functions and lazy processing can be a large performance improvement. A generator function can use less memory than a large data collection, and this change can be helpful. This book will provide number of examples of lazy processing to avoid creating large, in-memory collections. It can be difficult to get started with type hints. The use case in this book show type hints in somewhat more complex real-world situations. If you enjoyed this interview, head over to check out Steven’s latest edition of Functional Python Programming, He is leveraging Python to implement microservices and ETL pipelines. His other titles with Packt Publishing include Python Essentials, Mastering Object-Oriented Python, Functional Python Programming, and Python for Secret Agents. What is the difference between functional and object oriented programming? Building functional programs with F# Seven wrongs don’t make the one right: Solving a problem with Functional Javascript
Read more
  • 0
  • 0
  • 25439

article-image-unlocking-excels-potential-extend-your-spreadsheets-with-r-and-python
Steven Sanderson, David Kun
17 Oct 2024
5 min read
Save for later

Unlocking Excel's Potential: Extend Your Spreadsheets with R and Python

Steven Sanderson, David Kun
17 Oct 2024
5 min read
Introduction Are you an Excel user looking to push your data analysis capabilities beyond the familiar cells and formulas? If so, you're about to embark on a transformative journey. With the integration of R and Python, you can elevate Excel into a powerhouse of advanced data analysis and visualization. In this blog post, inspired by the book "Extending Excel with Python and R," co-authored by myself and David Kun, we will dive deep into practical implementation, focusing on how to automate data visualization in Excel using these powerful programming languages. Practical Implementation: Creating Advanced Data Visualizations In the world of data analysis, visual representation is key to understanding complex datasets. Excel, while equipped with basic charting tools, often requires enhancement for more sophisticated visuals. By integrating R and Python, you can create dynamic and detailed graphs that bring your data to life. Task: Automating Data Visualization with Python and R Step-by-Step Guide Step 1: Set Up Your Environment Before jumping into visualization, ensure you have the necessary tools installed. You will need: Excel: Ensure you have a version that supports VBA (Visual Basic for Applications). Python: Install Python on your computer. You can download it from the official Python website. R: Similarly, install R from the Comprehensive R Archive Network (CRAN). Libraries: For Python, install `pandas`, `matplotlib`, and `openpyxl` using pip. For R, install `ggplot2` and `readxl`.  Step 2: Importing Data Begin by importing your Excel data into Python or R. Here’s a Python snippet using pandas:  In R, use readxl:  Step 3: Creating Visualizations Python Example Using Matplotlib, you can create a simple line plot: Python Example   R Example With ggplot2, the process is equally straightforward where df is some data frame loaded in:  Step 4: Integrating Visualizations into Excel Once your visualization is created, the next step is to integrate it back into Excel. This can be done manually, or you can automate it using VBA or an API endpoint. Python Integration Using openpyxl, you can embed images:   R Integration For R, you might automate this process using R scripts that interact with Excel via VBA or other packages like `officer`.  Step 5: Automating the Entire Workflow To automate, consider using Python scripts executed from Excel VBA or R scripts called through Excel's RExcel plugin. This way, you can refresh data and update visualizations with minimal effort. Conclusion By integrating R and Python with Excel, you unlock a realm of possibilities for data visualization and analysis, turning Excel from a simple spreadsheet tool into a comprehensive data analytics suite. This guide provides a snapshot of what you can achieve, and with further exploration, the potential is limitless. Author Bio Steven Sanderson is a Manager of Applications with a deep passion for data and its compliments: cleaning, analysis, visualization and communication. He is known primarily for his work in R. After his MPH, Steven continued his work in the healthcare industry as a clinical decision support analyst working his way up to Manager of Applications at Stony Brook Medicine for Patient Financial Services. He currently is focused on expanding functions in his healthyverse suite of packages while also slimming them down and expanding their robustness. He also now enjoys helping mentor junior employees to set them up for success. David Kun is a mathematician and actuary who has always worked in the gray zone between quantitative teams and ICT, aiming to build a bridge. He is a co-founder and director of Functional Analytics, the creator of the ownR infinity platform. As a data scientist, he also uses ownR for his daily work. His projects include time series analysis for demand forecasting, computer vision for design automation, and visualization. Looking to Master Excel with Python and R?If you're excited about extending Excel’s capabilities with powerful tools like Python and R, Extending Excel with Python and R, authored by Steven Sanderson, David Kun, offers an in-depth guide to seamlessly integrating these languages into your Excel workflow. It covers everything from automating data tasks to advanced visualizations, all tailored for Excel enthusiasts.
Read more
  • 0
  • 0
  • 25315

article-image-instead-of-data-scientists-working-on-their-models-and-advancing-ai-they-are-spending-their-time-doing-deepops-work-missinglink-ceo-yogi-taguri-interview
Amey Varangaonkar
08 Nov 2018
10 min read
Save for later

“Instead of data scientists working on their models and advancing AI, they are spending their time doing DeepOps work”, MissingLink CEO, Yosi Taguri [Interview]

Amey Varangaonkar
08 Nov 2018
10 min read
Machine learning has shown immense promise across domains and industries over the recent years. From helping with the diagnosis of serious ailments to powering autonomous vehicles, machine learning is finding useful applications across a spectrum of industries. However, the actual process of delivering business outcomes using machine learning currently takes too long and is too expensive, forcing some businesses to look for other less burdensome alternatives. MissingLink.ai is a recently-launched platform to fix just this problem. It enables data scientists to spend less time on the grunt work by automating and streamlining the entire machine learning cycle, giving them more time to apply actionable insights gleaned from the data. Key Takeaways Processing and managing the sheer volume of data is one of the key challenges that today’s AI tools face Yosi thinks the idea of companies creating their own machine learning infrastructure doesn’t make a lot of sense. Data professionals should be focusing on more important problems within their organizations by letting the platform take care of the grunt work. MissingLink.ai is an innovative AI platform born out of the need to simplify AI development, by taking away the common, menial data processing tasks from data scientists and allowing them to focus on the bigger data-related issues; through experiment management, data management and resource management. MissingLink is a part of the Samsung NEXT product development team that aims to help businesses automate and accelerate their projects using machine learning We had the privilege of interviewing Mr. Yosi Taguri, the founder and CEO of MissingLink, to know more about the platform and how it enables more effective deep learning. What are the biggest challenges that companies come across when trying to implement a robust Machine Learning/Deep Learning pipeline in their organization? How does it affect their business? The biggest challenge, simply put, is that today’s AI tools can’t keep up with the amount of data being produced. And it’s only going to get more challenging from here! As datasets continue to grow, they will require more and more compute power, which means we risk falling farther behind unless we change the tools we’re using. While everyone is talking about the promise of machine learning, the truth is that today, assessing data is still too time-consuming and too expensive. Engineers are spending all their time managing the sheer volume of data, rather than actually learning from it and being empowered to make changes. Let’s talk about MissingLink.ai, the platform you and your team have launched for accelerating deep learning across businesses. Why the name MissingLink? What was the motivation to launch this platform? The name is actually a funny story, and it ties pretty neatly into why we created the platform. When we were starting out three years ago, deep learning was still a relatively new concept and my team and I were working hard to master the intricacies of it. As engineers, we primarily worked with code, so to be able to solve problems with data was a fascinating new challenge for us. We quickly realized that deep learning is really hard and moves very, very slow. So we set out to solve that problem of how to build really smart machines really fast. By comparison, we thought of it through the lens of software development. Our goal was to accelerate from a glacial pace to building machine learning algorithms faster -- because we felt that there was something missing, a missing link if you will. MissingLink is a part of the growing Samsung NEXT product development team. How does it feel? What role do you think MissingLink will play in Samsung NEXT’s plans and vision going forward? Samsung NEXT’s broader mission is to help startups reach their full potential and achieve their goals. More specifically, Samsung NEXT discovers and backs the engineers, innovators, builders, and entrepreneurs who will help Samsung define the future of software and services. The Samsung NEXT product development team is focused on building software and services that take advantage of and accelerate opportunities related to some of the biggest shifts in technology including automation, supply and demand, and interfaces. This will require hardware and software to seamlessly come together. Over the past few years, nearly all startups are leveraging AI for some component of their business, yet practical progress has been slower than promised. MissingLink is a foundational tool to enable the progress of these big changes, helping entrepreneurs with great use cases for machine learning to accelerate their projects. Could you give us the key features of Missinglink.ai that make it stand out from the other AI platforms available out there? How will it help data scientists and ML engineers build robust, efficient machine learning models? First off, MissingLink.ai is the most comprehensive AI platform out there. It handles the entire deep learning lifecycle and all its elements, including code, data, experiments, and resources. I’d say that our top features include: Experiment Management: See and compare the entire history of experiments. MissingLink.ai auto-documents every aspect Data Management: A unique data store tracks data versions used in every experiment, streams data, caches it locally and only syncs changes Resources Management: Manages your resources with no extra infrastructure costs using your AWS or other cloud credentials. It grows and shrinks your cloud resources as needed. These features, together with our intuitive interface, really put data scientists and engineers in the driver's seat when creating AI models. Now they can have more control and spend less energy repeating experiments, giving them more time to focus on what is important. Your press release on the release of MissingLink states “the actual process of delivering business outcomes currently takes too long and it is too expensive. MissingLink.ai was born out of a desire to fix that.” Could you please elaborate on how MissingLink makes deep learning less expensive and more accessible? Companies are currently spending too much time and devoting too many resources to the menial tasks that are necessary for building machine learning models. The more time data scientists spend on tasks like spinning machines, copying files and DevOps, the more money that a company is wasting. MissingLink changes that through the introduction of something we’re calling DeepOps or deep learning operations, which allows data scientists to focus on data science and let the machine take care of the rest. It’s like DevOps where the role is about how to make the process of software development more efficient and productionalized, but the difference is no one has been filling this role and it’s different enough that its very specific to the task of deep learning. Today, instead of data scientists working on their models and advancing AI, they are spending their time doing this DeepOps work. MissingLink reduces load time and facilitates easy data exploration by eliminating the need to copy files through data-management in a version-aware data store. Most of the businesses are moving their operations on to the cloud these days, with AWS, Azure, GCP, etc. being their preferred cloud solutions. These platforms have sophisticated AI offerings of their own. Do you see AI platforms such as MissingLink.ai as a competition to these vendors, or can the two work collaboratively? I wouldn’t call cloud companies our competitors; we don’t provide the cloud services they do, and they don’t provide the DeepOps service that we do. Yes, we all are trying to simplify AI, but we’re going about it in very different ways. We can actually use a customer’s public cloud provider as the infrastructure to run the MissingLink.ai platform. If customers provide us with their cloud credentials, we can even manage this for them directly. Concepts such as Reinforcement Learning and Deep Learning for Mobile are getting a lot of traction these days, and have moved out of the research phase into the application/implementation phase. Soon, they might start finding extensive business applications as well. Are there plans to incorporate these tools and techniques in the platform in the near future? We support all forms of Deep Learning, including Reinforcement Learning. On the Deep Learning for Mobile side, we think the Edge is going to be a big thing as more and more developers around the world are exposed to Deep Learning. We plan to support it early next year. Currently, data privacy and AI ethics have become a focal point of every company’s AI strategy. We see tech conglomerates increasingly coming under the scanner for ignoring these key aspects in their products and services. This is giving rise to an alternate movement in AI, with privacy and ethics-centric projects like Deon, Vivaldi, and Tim Berners-Lee’s Solid. How does MissingLink approach the topics of privacy, user consent, and AI ethics? Are there processes/tools or principles in place in the MissingLink ecosystem or development teams that balance these concerns? When we started MissingLink we understood that data is the most sensitive part of Deep Learning. It is the new IP. Companies spend 80% of their time attending to data, refining it, tagging it and storing it, and therefore are reluctant to upload it to a 3rd party vendor. We have built MissingLink with that in mind - our solution allows customers to simply point us in the direction of where their data is stored internally, and without moving it or having to access it as a SaaS solution we are able to help them manage it by enabling version management as they do with code. Then we can stream it directly to the machines that need the data for processing and document which data was used for reproducibility. Finally, where do you see machine learning and deep learning heading in the near future? Do you foresee a change in the way data professionals work today? How will platforms like MissingLink.ai change the current trend of working? Right now, companies are creating their own machine learning infrastructure - and that doesn’t make sense. Data professionals can and should be focusing on more important problems within their organizations. Platforms like MissingLink.ai free data scientists from the grunt work it takes to upkeep the infrastructure, so they can focus on bigger picture issues. This is the work that is not only more rewarding for engineers to work on, but also creates the unique value that companies need to compete.  We’re excited to help empower more data professionals to focus on the work that actually matters. It was wonderful talking to you, and this was a very insightful discussion. Thanks a lot for your time, and all the best with MissingLink.ai! Read more Michelangelo PyML: Introducing Uber’s platform for rapid machine learning development Tesseract version 4.0 releases with new LSTM based engine, and an updated build system Baidu releases a new AI translation system, STACL, that can do simultaneous interpretation
Read more
  • 0
  • 0
  • 25226
article-image-imran-bashir-on-the-fundamentals-of-blockchain-its-myths-and-an-ideal-path-for-beginners
Expert Network
15 Feb 2021
5 min read
Save for later

Imran Bashir on the Fundamentals of Blockchain, its Myths, and an Ideal Path for Beginners

Expert Network
15 Feb 2021
5 min read
With the invention of Bitcoin in 2008, the world was introduced to a new concept, Blockchain, which revolutionized the whole of society. It was something that promised to have an impact upon every industry. This new concept is the underlying technology that underpins Bitcoin.  Blockchain technology is the backbone of cryptocurrencies, and it has applications in finance, government, media, and many other industries.   Some describe blockchain as a revolution, whereas another school of thought believes that it is going to be more evolutionary, and it will take many years before any practical benefits of blockchain reach fruition. This thinking is correct to some extent, but, in Imran Bashir’s opinion, the revolution has already begun. It is a technology that has an impact on current technologies too and possesses the ability to change them at a fundamental level.  Let’s hear from Imran on fundamentals of blockchain technology, its myths and his recent book, Mastering Blockchain, Third Edition. What is blockchain technology? How would you describe it to a beginner in the field? Blockchain is a distributed ledger which runs on a decentralized peer to peer network. First introduced with Bitcoin as a mechanism that ensures security of the electronic cash system, blockchain has now become a prime area of research with many applications in a variety of industries and sectors.   What should be the starting point for someone aiming to begin their journey in Blockchain? Focus on the underlying principles and core concepts such as distributed systems, consensus, cryptography, and development using no helper tools in the start. Once you understand the basics and the underlying mechanics, then you can use tools such as truffle or some other framework to make your developer life easier, however it is extremely important to learn the underlying concepts first.   What is the biggest myth about blockchain? Sometimes people believe that blockchain IS cryptocurrency, however that is not the case. Blockchain is the underlying technology behind cryptocurrencies that ensures the security, and integrity of the system and prevents double spends. However, cryptocurrency can be considered one application of blockchain technology out of many.      “Blockchain is one of the most disruptive emerging technologies today.” How much do you agree with this? Indeed, it is true.  Blockchain is changing the way we do business. In the next 5 years or so, financial systems, government systems and other major sectors will all have blockchain integrated in one way or another.   What are the factors driving development of the mainstream adoption of Blockchain? The development of standards, interoperability efforts, and consortium blockchain are all contributing towards mainstream adoption of blockchain. Also demand for more security, transparency, and decentralization in some sectors are also key drivers behind more adoption, e.g., a prime solution for decentralized sovereign identity is blockchain.   How do you explain the term bitcoin mining? Mining is a colloquial term used to describe the process of creating new bitcoins where a miner repeatedly tries to find a solution to a math puzzle and whoever finds it first wins the right to create new block and earn bitcoins as a reward.    How can Blockchain protect the Global economy? I think with the trust, transparency and security guarantees provided by blockchain we can perceive a future where financial crime can be limited to a great degree. That can have a good impact on the global economy. Furthermore, the development of CDBCs (central bank digital currencies) are expected to have a major impact on the economy and help to stabilize it. From an inclusion point of view, blockchain can allow unbanked populations to play a role in the global financial system. If cryptocurrencies replace the current monetary system, then because of the decentralized nature of blockchain, major cost savings can be achieved as no intermediaries or banks will be required, and a peer to peer, extremely low cost, global financial system can emerge which can transform the world economy. The entire remittance ecosystem can evolve into an extremely low cost, secure, real-time system which can include people who were porously unbanked. The possibilities are endless.   Tell us a bit about your book, Mastering Blockchain, Third Edition? Mastering Blockchain, Third Edition is a unique combination of theory and practice. Not only does it provides a holistic view of most areas of blockchain technology, it also covers hands on exercises using Ethereum, Bitcoin, Quroum and Hyperledger to equip readers with both theory and practical knowledge of blockchain technology. The third edition includes four new chapters on hot topics such as blockchain consensus, tokenization, Ethereum 2 and Enterprise blockchains.  About the author  Imran Bashir has an M.Sc. in Information Security from Royal Holloway, University of London, and has a background in software development, solution architecture, infrastructure management, and IT service management. He is also a member of the Institute of Electrical and Electronics Engineers (IEEE) and the British Computer Society (BCS). Imran has extensive experience in both the public and financial sectors, having worked on large-scale IT projects in the public sector before moving to the financial services industry. Since then, he has worked in various technical roles for different financial companies in Europe's financial capital, London. 
Read more
  • 0
  • 0
  • 25174

article-image-minko-gechev-developers-should-learn-all-front-end-frameworks-to-go-to-the-next-level
Richard Gall
04 Jun 2018
6 min read
Save for later

Minko Gechev: "Developers should learn all major front end frameworks to go to the next level"

Richard Gall
04 Jun 2018
6 min read
This year's Skill Up survey produced some interesting results when it came to the best front end frameworks. Angular remains the most established tool with 40% of web developers reporting that they used it regularly. React is actually a little further behind, with 25% using it regularly. Similarly, Vue.js is growing but used by 20% of respondents. However, opinion was a little different when we asked what front end frameworks should win the battle of the 3 big front end tools. Respondents were split on Angular and React, with both JavaScript tools winning 34% of the vote. Vue wasn't far behind, at just over 30%. With the web development world apparently split over what framework is going to define the future of the field, how are we to pick them apart? Or do we even really need to worry? Read the report in full. Sign up to our weekly newsletter and download the PDF for free. The fact that we have three great front end tools jostling for developer attention is surely a good thing, right? To help us make sense of these trends, we caught up with Angular expert Minko Gechev to find out what he makes of web development in 2018, and what front end developers should be learning. Minko Gechev is the author of Switching Angular. You can find the latest edition here on the Packt Store. Which front end framework should you learn: Angular, React, or Vue? Respondents to the Skill Up survey were evenly split between Angular, React, and Vue in the 'battle of the frameworks'. Which do you think developers should learn, and why? In all of them, there are unique and interesting ideas which are worth exploring. I truly believe that learning all the major frameworks can help developers go to the next level! This doesn’t necessary mean to be proficient in all of them. Having a high-level understanding of how the frameworks work and how to use them is completely enough and will allow you to adapt according to the projects’ requirements. This is similar to learning programming languages from different paradigms – it helps you discover how problems are being solved in different ways. For the past a couple of years, the redux pattern became the de facto standard for state management in the modern front-end development. The good thing about redux is that it’s view agnostic so you can use it with any framework – Vue, React, Angular, etc. Angular has its own redux alternative called ngrx which empowers a declarative approach with RxJS but in general, it follows the same underlying pattern. My recommendation would be to understand how to manage the state of our applications because that’s probably the most complex problem that we’re solving in our day to day development process. Once we have a solid understanding of this, we can easily switch between different frameworks depending on the problems we’re solving, what the rest of the team is using, and the project’s requirements. A very interesting characteristic of learning Angular is that if we get comfortable with the framework we’d be also familiar with TypeScript, RxJS, and techniques such as dependency injection. This may look like an initial overhead but it’s a great long-term investment which pays off really well in large projects. How important is TypeScript to front end development? How important is TypeScript to modern web development? Why? Over the past a couple of years I see a strong increase in the excitement around the language, not only in the Angular world but also in React and Vue. I’m personally using TypeScript for a few projects – a platform that we built with React and an educational application written in Angular. I see a lot of value in using TypeScript. Recently I haven’t started any project with JavaScript – for everything new I’m using TypeScript and I’m trying to migrate, as many of my existing projects as possible.There are a few reasons for this: TypeScript provides great development experience! Especially, combined with VSCode, you can instantly notice when you’ve misspelled a property, method, you’re trying to access a property of a nullable value, etc. It gives you a sense of security that your program is correct to given extent. Of course, TypeScript cannot save us from logical mistakes but if we use its type system wisely, we can get great benefits. You might be curious what benefits? Well, TypeScript can help us reduce the number of bugs in our programs. In the study “To Type or Not to Type: Quantifying Detectable Bugs in JavaScript” the authors shown that the average JavaScript program can benefit with 15% bug reduction if it uses the type system of TypeScript. The study was using TypeScript version 2.0; with the latest features of the language the number of detectable bugs is growing dramatically. Since recently webpack is leveraging TypeScript as well because it helps discover already existing issues in the codebase. Web developers and JavaScript fatigue Do you think we're past web developers experiencing 'JavaScript fatigue'? JavaScript is very dynamic and it moves very quickly. There are a lot of potential issues which could be caused by a variety of reasons. With semantic versioning and powerful type systems (such as the type system of TypeScript), we’re walking in the right direction but we definitely have a long way to go until the ecosystem matures. Web development over the next 12 months: WebAssembly and machine learning What do you think will be the most important thing for developers to learn in the next 12 months? There are a lot of exciting things happening nowadays! Web browsers are getting more and more powerful, exposing hundreds of APIs and opportunities. WebAssembly is moving very quickly and I believe that together with Rust it has a lot of potential in future. On the other hand, Google recently announced TensorFlow.js. This is a library which allows us to use machine learning (ML) in the browser. In the next years, ML is going to take a larger portion of our development process (directly or indirectly) for: Implementing features in our applications Improving development process I’m specifically interested in the second point – improving our development process by using ML. Together with Addy Osmani, Kyle Mathews, and Katie Hempenius, we’ve been working on a toolkit called Guess.js. It aims to provide predictive bundling and pre-fetching based on ML techniques in order to let us develop faster Angular/React/Vue/etc. applications. I’m really excited about what’s coming up in near future! So are we! Thanks for taking the time to speak to us, Minko!
Read more
  • 0
  • 0
  • 24797

article-image-listen-we-discuss-what-it-means-to-be-a-hacker-with-adrian-pruteanu-podcast
Richard Gall
26 Apr 2019
2 min read
Save for later

Listen: We discuss what it means to be a hacker with Adrian Pruteanu [Podcast]

Richard Gall
26 Apr 2019
2 min read
With numerous high profile security breaches in recent years, cybersecurity feels like a particularly urgent issue. But while the media - and, indeed, the wider world - loves stories of modern vulnerabilities and mischievous hackers, there's often very little attention paid to what causes insecurity and what can practically be done to solve such problems. To get a better understanding of cybersecurity in 2019, we spoke to Adrian Pruteanu, consultant and self-identifying hacker. He told us about what he actually does as a security consultant, what it's like working with in-house engineering teams, and how red team/blue team projects work in practice. Adrian is the author of Becoming the Hacker, a book that details everything you need to know to properly test your software using the latest pentesting techniques.          What does it really mean to be a hacker? In this podcast episode, we covered a diverse range of topics, all of which help to uncover the reality of working as a pentester. What it means to be a hacker - and how it's misrepresented in the media The biggest cybersecurity challenges in 2019 How a cybersecurity consultant actually works The most important skills needed to work in cybersecurity The difficulties people pose when it comes to security Listen here: https://soundcloud.com/packt-podcasts/a-hacker-is-somebody-driven-by-curiosity-adrian-pruteanu-on-cybersecurity-and-pentesting-tactics
Read more
  • 0
  • 0
  • 24731
article-image-how-gremlin-is-making-chaos-engineering-accessible-interview
Richard Gall
14 Jun 2018
10 min read
Save for later

How Gremlin is making chaos engineering accessible [Interview]

Richard Gall
14 Jun 2018
10 min read
Despite considerable hype, chaos engineering doesn’t appear to have yet completely captured the imagination of the wider software engineering world. According to this year’s Skill Up survey, when asked, only 13% of developers said they were excited about it. But that doesn’t mean we should disregard - far from it. Like many of the best trends, it might blow up when we least expect. It might find its way onto your CTOs eyes in just a few months. As site reliability engineering grows as a discipline, and as businesses start to put a value on downtime, chaos engineering is likely to become a big part of the reliability and resilience toolkit. Gremlin, chaos engineering, and the end of the age of downtime “People are expected to always be up” says Matt Fornaciari, co-founder and CTO of Gremlin, a product that offers “failure as a service” to businesses. I spoke to Fornaciari last month to get a deeper insight on Gremlin and the team and ideas behind it. He believes the world has changed in recent years, and the days of service windows when sites would just be taken down for an hour or two for an update or change is over: “that’s unacceptable to people these days.” Fornaciari isn’t an unbiased observer, of course. The success of Gremlin depends on chaos engineering’s adoption and acceptance. However, he’s not going out on a limb; there’s clear VC interest in Gremlin. At the end of 2017 the company received their first round of funding - more than 7 million USD. It’s a cliche but money does talk - and in this instance it seems to be saying that this approach might change the way we think about building our software. Arguably, chaos engineering - and by extension Gremlin - is a response to other trends in software. “I’ve seen a lot of signals that this is the way the world’s going”, Fornaciari says. He’s referring here to broader trends like cloud and microservices. He explains that because microservices is all about modularity, and breaking aspects of your software infrastructure into smaller pieces “you end up with nodes in this network” which “adds network complexity.” Consequently, this additional complexity means there is more that can go wrong - it becomes more unreliable. Gremlin’s bid to democratize chaos engineering It’s important to note here that chaos engineering has been around for some time - it’s not a radically new methodology. But it’s largely been locked away in some of the world’s biggest tech companies, like Netflix and Amazon. Many of Gremlin’s leaders actually worked at those companies - Fornaciari has worked at Salesforce and Amazon, for example. “The main goal was to democratize chaos engineering… we’ve [the Gremlin team] done it at the bigger companies and we’re like you know what, everyone can benefit from this”. That is the essential point around chaos engineering. If it’s going to catch on in the mainstream tech world, it needs to be more accessible to different businesses. Fornaciari explains that many of Gremlin’s customers are larger organizations. These are companies for whom downtime is of utmost importance, where a site outage that lasts just an hour could cost thousands of dollars. That said, from a cultural perspective, many organizations find it difficult to adopt this sort of mindset. “Proving the value of something that doesn’t happen,” Fornaciari says, is one of the biggest challenges for Gremlin. This is particularly true when selling their tool. Pager pain: How Gremlin sells chaos engineering to customers This is how Gremlin does it: “We have three qualifying questions: do you measure your downtime? Do you have somebody who’s responsible for downtime? And do you actually have a dollar amount tied to it?” Presumably, for many organizations at least one answer to these questions is “no”. That’s why customer support is so important for Gremlin. “Customer success and developer advocacy are two of our biggest initiatives… I’ve told people as we’re recruiting them that half of our goal as a company is to educate people.” Gremlin’s challenges as a product and as a business reflect the wider difficulties of managing upwards. The tension between those ‘on the ground’ and those at a more senior and managerial level is one that Gremlin is acutely aware of. This is where a lot of push back comes from, Fornaciari explains: What we’ve seen so far is just push back from top down - like, why do we need this? We use the term pager pain to define the engineer on call - the closer you are to the ground the closer you are to the on call rotation and the more you feel those pains and the more you believe in this but as you raise up a couple of levels you maybe don’t feel that as much… if you don’t have that measure on uptime - unless someone is on the hook for that at a higher level there’s oftentimes a why do we need this, why are we going to spend money on breaking things. Pager pain is a nice concept - it captures the tension between different layers of management. It highlights the conflict between ‘what do we need?’ and ‘what can we do?’ Read next: Blockchain can solve tech's trust issues  Safety, simplicity and security To successfully sell Gremlin, the way the product is designed is everything. For that reason, the Gremlin team have three tenets built into their product: safety, security, simplicity. When you’ve got a “potentially dangerous tool,” as Fornaciari himself describes it, making sure things are safe and secure is absolutely essential. Arguably, the fact that chaos engineering is so hard to do well might be something that Gremlin can use to its advantage. “One thing we hear when we talk to companies about it is ‘well we’ll go build this ourselves’ and the fact is it’s a really hard thing to do, and a hard thing to do well.” Gremlin is walking on a bit of a tightrope. On the one hand chaos engineering is for everyone, but on the other it’s difficult and dangerous. It should be accessible, but not too accessible. “One of the reasons we don’t have a free offering is because we are a little worried about protecting our customers not doing any harm to people… I mean, this is essentially giving somebody a potentially dangerous tool.. If they’re not given the proper education then that could be a problem, right?” Gremlin aren’t the only chaos engineering product out there. As with any trend, there are plenty of software platforms and tools emerging for technologically forward thinking businesses. Fornaciari doesn’t see these as a threat - he’s confident, bullish even, about Gremlin’s place in the market. “There are a lot of tools out there that people can go and use but they really lack the safety and simplicity.” Alongside its philosophy of safety, security and simplicity, a big selling point, according to Fornaciari, is the experience and expertise that is built into Gremlin’s DNA. “We’ve got fifteen years of combined expertise in this space” he says. “Being the experts on it and having built it 3 or 4 times already in different big companies, it sort of gave us this leg up to go out there in the world.” But while Fornaciari is eager to assert Gremlin’s knowledge, there’s no trace of elitism - sharing knowledge is a core part of the product offering. “We actually built out customer success tooling so we can see if particular attacks fail for them we can actually proactively reach out and be like ‘hey we saw you were trying to do this, maybe you meant to do this’” Fornaciari explains. Controlled chaos: chaos engineering and the scientific method Control is central to Gremlin’s philosophy - it’s a combination of the team’s commitment to safety, security and simplicity. In fact, this element of control that distinguishes chaos engineering today, from what went before. Central to Gremlin’s mission to make chaos engineering accessible, is also redefining how it’s done. “If you’re familiar with the netflix chaos monkey mentality of randomly terminating services, well that’s a good start, but safety is really lacking. We talked more about this controlled chaos… this idea that you start fairly small with this small blast radius and then as you become more confident you grow it out and grow it out as opposed to just like ‘cool, let’s just chuck a grenade in here and see what happens.’” Fornaciari goes on to describe this ‘controlled chaos’ in a surprising way. “It’s much more like the scientific method actually. Applying that method to your infrastructure and your reliability in general.” This approach is essential if you’re going to do chaos engineering well. How to do chaos engineering effectively When I ask Fornaciari how engineering teams and businesses can do chaos engineering well he emphasizes the importance of starting with a hypothesis: “You need to have a hypothesis that you’re trying to prove.Throwing random chaos at something is fine - it’ll sort of surface some of the unknown unknowns for you. But really having a hypothesis that you’re trying to prove is the best way to get value out of this [chaos engineering].” If you’re going to take a scientific approach to testing your infrastructure using ‘chaos experiments’, managing scale is also incredibly important. Don’t run before you can walk is the message. “Keep it very small initially, then you start to grow the blast radius. You definitely want to make sure that you’re starting off with the smallest modicum that you can.” Given the potential dangers of throwing metaphorical gremlins into your system, starting where your comfortable makes a lot of sense. “Start in staging, start where your comfortable, build your confidence. Make sure your system behaves well in front of non-customer facing traffic before you go out to the world.” That said, Gremlin have had “some pretty bold customers” who go straight ahead and start running chaos experiments in production. “That was cool. It’s a little scary, but they were confident and they’ve been using Gremlin as part of their system ever since.” Chaos engineering requires confidence and control Ultimately, if chaos engineering is going to take off - as Fornaciari believes it will - engineers will need to be incredibly confident. That’s true on a number of levels. You need confidence that you’ll be able to handle a range of experiments and deploy them wisely. But you’ll also need confidence that you can manage the expectations of those in senior management. It’s not hard to see the value of chaos engineering. As Fornaciari says “if you prevent one outage one time, you’ve saved that money to pay for the tool to make sure it doesn’t happen again.” But it might be hard to find time for it. It might be hard to get buy in and investment in the tools you need to do it. Gremlin are certainly going to play an important part in helping engineers do that. But one of its biggest challenges - and perhaps one of its most noble missions too - is transforming a culture where people don’t really appreciate ‘pager pain’. If Fornaciari and Gremlin can help solve that, good luck to them. You can follow Matt Fornaciari on Twitter: @callmeforni
Read more
  • 0
  • 0
  • 24308

article-image-cybersecurity-researcher-elliot-alderson-talks-trump-and-facebook-google-and-huawei-and-teaching-kids-online-privacy-podcast
Richard Gall
08 Aug 2019
3 min read
Save for later

Cybersecurity researcher "Elliot Alderson" talks Trump and Facebook, Google and Huawei, and teaching kids online privacy [Podcast]

Richard Gall
08 Aug 2019
3 min read
For anyone that's watched Mr. Robot, the name Elliot Alderson will sound familiar. However, we're not talking about Rami Malek's hacker alter ego - instead, the name has been adopted as an alias by a real-life white-hat hacker who has been digging into the dark corners of the wild and often insecure web. Elliot's real name is Baptiste Robert (whisper it...) - he was kind enough to let us peak beneath the pseudonym, and spoke to us about his work as a cybersecurity researcher and what he sees as the biggest challenges in software security today. Listen: https://soundcloud.com/packt-podcasts/cybersecurity-researcher-elliot-alderson-on-fighting-the-good-fight-online "Elliot Alderson" on cybersecurity, politics, and regulation In the episode we discuss a huge range of topics, including: Security and global politics Is it evolving the type of politics we have? Is it eroding trust in established institutions? Google’s decision to remove its apps from Huawei devices The role of states and the role of corporations Who is accountable? Who should we trust? Regulation Technological solutions What Elliot Alderson has to say on the podcast episode... On Donald Trump's use of Facebook in the 2016 presidential election: “We saw that social networks have an impact on elections. Donald Trump was able to win the election because of Facebook - because he was very aggressive on Facebook and able to target a lot of people…”  On foreign interference in national elections: “We saw, also, that these tools… have been used by countries… in order to manipulate the elections of another country. So as a technician, as a security researcher, as an infosec professional, you need to ask yourself what is happening - can we do something against that? Can we create some tool? Can we fight this phenomenon?” How technology professionals and governing institutions should work together: “We should be together. This is the responsibility of government and countries to find vulnerabilities and to ensure the security of products used by its citizens - but it’s also the responsibility of infosec professionals and we need to work closely with governments to be sure that nobody abuses vulnerabilities out there…” On teaching the younger generation about privacy and protecting your data online: “I think government and countries should teach young people the value of personal data… personally, as a dad, this is something I’m trying to teach my kids - and say okay, this website is asking you your personal address, your personal number, but do they need it? ...In a lot of cases the answer is quite obvious: no, they don’t need it.” On Google banning Huawei: “My issue with the Huawei story and the Huawei ban is that as a user, as a citizen, we are only seeing the consequences. Okay, Google ban Huawei - Huawei is not able to use Google services. But we don’t have the technical information behind that.” On the the importance of engineering ethics: “If your boss is coming to you and saying ‘I would like to have an application which is tracking people during their day to day work’ what is your decision? As developers, we need to say ‘no: this is not okay. I will not do this kind of thing’”. Read next: Doteveryone report claims the absence of ethical frameworks and support mechanisms could lead to a ‘brain drain’ in the U.K. tech industry Follow Elliot Alderson on Twitter: @fs0c131y
Read more
  • 0
  • 0
  • 24216
Modal Close icon
Modal Close icon