Tech News - Databases

I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. Today’s tip is to ask yourself, will this still matter a year from now? A year is a long time. Next year, as we get ready for Halloween, will this matter? I certainly hope that much of the challenges of COVID will be a memory. While I wouldn’t be surprised if we still wore masks in gatherings and limited some of the heavily crowded areas, I do hope that we have learned some lessons and adjusted our world to cope. The feelings I have today, and discontent, the stress and concerns, much if it won’t matter. The post Daily Coping 30 Oct 2020 appeared first on SQLServerCentral.

Workflow Tooling Development workflow, especially outside of a full-fledged IDE, is often a disjointed affair. DevOps oriented workflows that often combine cli tools such as terraform, PowerShell, bash, and more all provide more complexity to getting up to speed and productive. Currently, there is a variety of frameworks to solve this problem. The “gold standard” most are familiar with in the open-source community would be Make. Considering Cross-Platform Tooling This is not an exhaustive list, it’s focused more on my journey, not saying that your workflow is wrong. I’ve looked at a variety of tooling, and the challenge has typically that most are very unintuitive and difficult to remember. Make…it’s everywhere. I’m not going to argue the merits of each tool as I mentioned, but just bring up that while cMake is cross platform, I’ve never considered Make a truly cross platform tool that is first class in both environments. InvokeBuild & Psake In the Windows world, my preferred framework would be InvokeBuild or PSake. The thing is, not every environment will always have PowerShell, so I’ve wanted to experiment with minimalistic task framework for intuitive local usage in a project when the tooling doesn’t need to be complex. While InvokeBuild is incredibly flexible and intuitive, there is an expectation of familarity with PowerShell to fully leverage. If you want a robust framework, I haven’t found anything better. Highly recommend examining if you are comfortable with PowerShell. You can generate VSCode tasks from your defined scripts and more. InvokeBuild & Psake aren’t great for beginners just needing to run some tooling quickly in my experience. The power comes with additional load for those not experienced in PowerShell. If you are needing to interact with AWS.Tools SDK, complete complex tasks such as generating objects from parsing AST (Abstract Syntax Trees) and other, then I’d lead towards InvokeBuild. However, if you need to initialize some local dependencies, run a linting check, format your code, get the latest from main branch and rebase, and other tasks that are common what option do you have to get up and running more quickly on this? Task I’ve been pleasantly surprised by this cross-platform tool based on a simple yaml schema. It’s written in go, and as a result it’s normally just a single line or two to immediately install in your system. Here’s why you might find some value in examining this. Cross-platform syntax using this go interpreter sh Very simple yaml schema to learn. Some very nice features that make it easy to ignore already built assets, setup task dependencies (that run in parallel too!), and simple cli interactivity. My experience has been very positive as I’ve found it very intuitive to build out basic commands as I work, rather than having to deal with more more complex schemas. Get Started version: 3 tasks: default: task --list help: task --list fmt: desc: Apply terraform formatting cmds: - terraform fmt -recursive=true The docs are great for this project, so I’m not going to try and educate you on how to use this, just point out some great features. First, with a quick VSCodee snippet, this provides you a quick way to bootstrap a new project with a common interface to run basic commands. Let’s give you a scenario… assuming you aren’t using an already built Docker workspace. I need to initialize my 2 terraform directories. I want to also ensure I get a few go dependencies for a project. Finally, I want to validate my syntax is valid among my various directories, without using pre-commit. This gets us started… version: 3 tasks: Next, I threw together some examples here. Initialize commands for two separate directories. A fmt command to apply standardized formatting across all tf files. Finally, wrap up those commands with a dep: [] value that will run the init commands in parallel, and once that is finished it will run fmt to ensure consistent formatting. version: '3' env: TF_IN_AUTOMATION: 1 tasks: init-workspace-foo: dir: terraform/foo cmds: - terraform init init-workspace-bar: dir: terraform/bar cmds: - terraform init fmt: desc: Recursively apply terraform fmt to all directories in project. cmds: - terraform fmt -recursive=true init: desc: Initialize the terraform workspaces in each directory in parallel. deps: [init-workspace-foo,init-workspace-bar] cmds: - task: fmt You can even add a task in that would give you a structured git interaction, and not rely on git aliases. sync: desc: In GitHub flow, I should be getting lastest from main and rebasing on it so I don't fall behind cmds: - git town sync Why not just run manually I’ve seen many folks online comments about why even bother? Can’t the dev just run the commands in the directory when working through it and be done with it? I believe tasks like this should be thrown into a task runner from the start. Yes, it’s very easy to just type terraform fmt, go fmt, or other simple commands… if you are the builder of that project. However: it increases the cognitive load for tedious tasks that no one should have to remember each time the project grows. It makes your project more accessible to new contributors/teammates. It allows you to simply moving to automation by wrapping up some of these automation actions in GitHub Actions or equivalent, but simply having the CICD tooling chosen run the same task you can run locally. Minimal effort to move it to automation from that point! I think wrapping up things with a good task runner tools considers the person behind you, and prioritizes thinking of others in the course of development. It’s an act of consideration. Choose the Right Tooling Here’s how I’d look at the choices: Run as much in Docker as you can. If simple actions, driven easily on cli such as build, formatting, validation, and other then start with Task from the beginning and make your project more accessible. If requirements grow more complex, with interactions with AWS, custom builds for Lambda, combined with other more complex interactions that can’t easily be wrapped up in a few lines of shell scripting… use InvokeBuild or equivalent. This gives you access to the power of .NET and the large module collection provided. Even if you don’t really need it, think of the folks maintaining or enabling others to succeed with contributions more easily, and perhaps you’ll find some positive wins there. ?? #development #cool-tools #golang #automation The post Improving Local Development Workflow With Go Task appeared first on SQLServerCentral.

Writing most of this Monday afternoon while on lunch break. I’m attending the Advanced T-SQL Querying and Query Tuning pre-con by Itzik Ben-Gan. You access pre-cons via the Virtual Summit web site or app, but in this case (idk, may all pre-cons) it’s presented via GotoWebinar. Quick notes on the experience: All users muted due to 100+ attendees 75 minute blocks followed by a 15 minute break I’m on the east coast, so for it’s noon to 8 pm and the “lunch” break at 2:45 pm (I ate during the 1:15 pm break and went for a walk during the lunch hour) Attendees can’t see questions or interact with each other. Sometimes the moderator will break in, but more its Itzik checking the question queue as he goes. At one point he tried dragging the question pane on his view over where we could see it, that sort of worked. It’s definitely a little different when he poses a question and usually you’d have people raising their hand or calling out the answer, you just wait and see what he reads back to you It’s the usual goto experience, not much to figure out or deal with Using Paint as the whiteboard, works ok Content wise, it is of course fine. As is often the case if you’re not a beginner the trick is to not zone out during the stuff you know so you don’t miss the stuff you don’t know. So far I learned three things, not bad. I suspect as it gets closer to 8 pm it’s going to be harder to stay attentive. I went back to the Summit site and visited all the sponsors. A very varied experience. Quite a few raffle options, one ebook, one that has free training (Red Hat), and of course opportunites to get free trials and demos. I’d encourage all the sponsors to look at what the other sponsors did. Most are vanilla and the pattern tends to encourage that, but it’s a bit of downer to see no sponsor prize or click a link that just takes you to a landing page. Still, I appreciate the sponsors more than ever this year and I’ll do my best to read whatever email they send me (the first couple anyway). There is a “grab bag” section which is, well, not exciting. There is a “comic poster” section that has stuff from various vendors. It’s worth a quick look. You can get them as a PDF, but I don’t see an option to order/buy one. Maybe this could be something, not sure, but would be a lot more interesting as a printed poster. Could also be more interesting if this was opened up to attendees (or even non-attendees) to contribute, though I don’t know that I’d want to be the one reviewing them. There is a section where you can find scheduled live demos from sponsors. I like that, it’s clear that it’s product focused. Looks like they are scheduled for an hour. I tagged one each day, will see how my schedule looks when I get there. Also, I might be on Twitter a bit this week. Definitely missing the people part of the Summit. Finally, I tried the information desk and couldn’t get audio to work, probably because I was logged in to gotowebinar, but was able to chat and found out they had 9 visitors and 10 questions so far on Monday, so people are finding it. The post Notes on the 2020 PASS Virtual Summit – Part 2 appeared first on SQLServerCentral.

I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. Today's tip is to add the word "yet" when you feel you can't do something. This is good advice because telling yourself that you can't do something before you even try is a self-fulfilling prophesy.  Accomplishing most goals require figuring out what's needed. Do you want to run a marathon, but you've never really run before? Set a smaller goal of running a 5K or even a mile.  If you make a good plan and follow through, you have a shot at that bigger goal.   The post Daily Coping 10 Nov 2020 appeared first on SQLServerCentral.

I’m in learning mode this week, scheduled for two full day pre-cons plus the Summit. I’ve been to a couple of virtual events already this year, but they were ones where I had never been to the in-person version so it was hard to compare things. For the Summit my goal is to try to experience as much as the UI allows just to see if there are things that really work or really don’t work. This post focuses on what I’ve seen so far, before any of the presentations start. If you haven’t checked in yet (or aren’t able to attend this year), you can get a good sense of how it works by watching this four minute video Anika did: Credentials were emailed, just your email address and a registration id that serves as the password. I don’t see an option to change the password. Not great, but not the worst issue either. You can login via the web, or via the app (https://cdmcd.co/Jx578w) which is EventScribe and seems to allow you to take notes as text or drawing. Logging in via phone worked fine and seems to be a similar UI to the web. On login you can update your profile, add a photo, add various ribbons, and choose from any or all of three sharing options: Share your contact information and profile with other attendees? Share your information with Sponsors/Exhibitors you interacted with? Share your favorites with other attendees? Living large, I said yes to all three! I then reviewed the main event schedule and favorited (clicked the star) quite a few sessions that I might attend. I didn’t try to build out a fully packed schedule as I don’t have to worry a session being at capacity (or so I assume!). The ones you favorite then show up as ‘your schedule’. Looking at the available sessions there are some interesting things: You can see if the slides are available All times are in EST (I’m assuming that’s because I’m in EST) There is video icon if the session is recorded (or will be?). Not all of the sessions are delivered live, and some are replays of a session delivered live earlier in the event. There isn’t an obvious signal that some sessions are given more than once. You can figure it out by clicking on the speaker name and that will show all of their sessions. Would be nice to see that in a simpler fashion when you’re making your scheduling choices. For recorded sessions speakers are supposed to be online for the final 15 minutes to take questions. I’m interested to see how that goes. There is a virtual exhibit hall. Here it is as of Monday morning: And here are a couple of examples: For the sponsor booth giveaway Redgate has a puzzle you have to figure out to be entered into a drawing. Quest will mail you a t-shirt if you enter your info and are one the first 500 (well, 499 now, since I entered for one). They have chat hours, you can see some slides, all the basics. I’m going to try to visit all of them, both for the experience of seeing how various vendors do things and to try to help them have a successful event. I can imagine it’s more than the average leap of faith to hope a virtual event proves as useful to a sponsor as an in person event. The pattern seems usable and one that seems like we could (and should) add to SQLSaturday. There is only one community hosted event listed so far this year, the SQLRun. That’s about as far as I’ve explored so far on the attendee experience. I’ll be the moderator for Continuous Integration with Local Agents and Azure DevOps at 1030 on Friday, presented by my friend Steve Jones. Last week I went to moderator training and got to see how it looks behind the scenes a little. Basically presentations are done via a Zoom call with the speaker, moderator, and some support staff. Attendees can submit questions or chat with each other via the ‘Discuss’ tab. Attendees can upvote questions. The moderator can answer questions, mark them as rejected, or hold them to try to give to the speaker at an appropriate time. The tricky part (and the unpleasant last minute surprise for all the live speakers) is that the presentation is captured in Zoom, sent to Cadmium (the streaming/conference platform) and from there delivered to users, which adds up to to a variable delay of 20 to 90 seconds. I suspect most attendees won’t notice because the questions are queued, but for speakers that had planned a highly interactive session it’s going to be a challenge. Registration is still open for the Summit at https://www.pass.org/summit/2020/. The post Notes on the 2020 PASS Virtual Summit – Part 1 appeared first on SQLServerCentral.

Alex is a Data Platform MVP who loves DevOps. He’s has been helping data professionals apply DevOps principles to relational database development and deployment since 2010. He’s most proud of helping Skyscanner develop the ability to deploy 95 times a day. Alex has worked with clients on every continent except Antarctica – so he’s keen to meet anyone who researches penguins. Source The post ASF 034: Alex Yates interview (part 1) appeared first on SQLServerCentral.

The Problem When you use a UDF in .NET for Apache Spark, something like this code: class Program { static void Main(string[] args) { var spark = SparkSession.Builder().GetOrCreate(); _logging.AppendLine("Starting Select"); var udf = Functions.Udf<int, string>(theUdf); spark.Range(100).Select(udf(Functions.Col("id"))).Show(); _logging.AppendLine("Ending Select"); Console.WriteLine(_logging.ToString()); } private static readonly StringBuilder _logging = new StringBuilder(); private static string theUdf(int val) { _logging.AppendLine($"udf passed: {val}"); return $"udf passed {val}"; } } Generally, knowing .NET we would expect the following output: The post Blog .NET for Apache Spark UDFs Missing Shared State appeared first on SQLServerCentral.

This month’s T-SQL Tuesday is hosted by my good friend and data platform MVP Taiob Ali. Taiob’s challenge is very relevant to current season – he asks to blog about how we are coping with what life has dealt us during this time. I really wondered where to start on this..because this year has stuck me with so many challenges that I never imagined I would face. I have been blessed with reasonably good health, longevity and good health in my family and a relatively low stress life other than periods which passed soon. But this year was not meant to be that. Clearly I had lessons to learn that I haven’t, and the universe decided it was time I did. My issues started well before the pandemic, around February or March, with some health challenges that seemed minor – some dental issues, digestive issues and female health related. I went through my annual physical, which came back good, and my doctor sent me home with some antacids, and also advised me to consult an orthodentist for my jaw related problems , which I had had ten years ago. I am relatively new to the Raleigh area, so had to look up a good orthodontist here, and that took a bit of time. I was still going to work as usual and coping, with the help of some OTCs and researching dentists as I went along. By March, Coronavirus had been declared a pandemic and our office decided to let us telecommute full time by mid April. I still recall the day – I had tuned a difficult stored procedure and taken off early. I was doing some shopping at Target when my boss messaged me to ask if I could come in and look into some timeouts the said procedure was causing on production. So I decided to purchase the (expensive) purse I was looking at (that purse will remain with me for a lifetime, as it was the last in-store shopping i would do for months). I went back in to work, fixed some issues on the procedure and headed home. It would be my last visit to the office for a long time. We were asked to work from home the next day onwards, and are on telecommute since then and will be for an indefinite period of time. Going to work was an activity i enjoyed – the office is literally one block from where I lived, the people and environment was very fun to be around. I started to feel really low (in addition to the general depression the pandemic was causing) being stuck at home all the time. I spent too much time dwelling on this and forgot all about the orthodontist search, which did not serve me well. By June, the jaw problems had escalated seriously and I could not eat anything without severe pain and distress. I was put on a jaw appliance and advised no solid food. I lost 10 lbs in two weeks and with great difficulty managed to find an odd combo of things to eat which did not hurt the jaw and provided some degree of nutrional support. The pain subsided in five weeks but only subject to the jaw appliance being in place. I am seeing another orthodontist to get further advice. But the combo of all this has led to a ton of health issues, esp digestion related. I won’t say much on that except that am getting treated and it will be slow but surely heal. In the meantime -I lost 3 beloved relatives to Covid. I am not good with emotions in general, and being stuck with so much was unbelievably stressful. Thankfully, the amazing therapist I had in Louisville who helped me cope with my mom’s untimely death was available on telemedicine. I was able to process some emotions and grief with his aid. I can’t say am still through with it but am able to manage and be functional. I can write a few more pages on my own story, but I think this is enough for people to get an idea. My lessons are as below..these are not things that I did not know but now I know really well and hope other people will consider: 1 Your body and mind are your vehicles to living and well being. Make it #1 priority to take care of them. Having a fun job that you enjoy does not mean you are doing this – a lot of us as techies think being absorbed in work means we are having a great life – what we neglect in the process can hit us hard, especially as we get older. Force yourself to log off when you are done, get that annual physical/exercise/dental appointment/nutritionist appointment in.2 Stay in touch with family, especially elders – make the best of the time you get with them, even if it is virtual. We don’t know how much we have left and it is good not to have any regrets around this.3 If you are going through chronic pain/discomfort, develop ways to focus on something – whatever gives your mind a way away from it. I take short walks, look at picture books with scenic pictures, old photo albums and various. 4 Get a therapist or a friend to do a healing visualizing meditation – there are lots of such meditations available but a personalized one , with scenarios that you want to happen are very helpful. I use the one my therapist made for me every day and it helps me, a lot. Some people who are religious may also be able to use prayer in this manner.5 Last but not the least, stay safe through it. This means a lot of stress in of itself. I know several friends and family membes who have gotten the virus, including some who are very healthy. Nobody thinks of it as trivial. Mask up and limit exposure to any place with people. ‘THIS TOO SHALL PASS AWAY’…is a wise line that has helped me many a time. It is a line to ponder repeatedly. We will get through, it will all be fine. Peace be with everyone. The post T-SQL Tuesday 132: Coping with a pandemic appeared first on SQLServerCentral.