It’s been an exciting week at endjin! We attended the Computing Rising Star Awards 2019 as both of our current Apprentices were shortlisted; Carmel Eve for Apprentice Engineer of the Year and Ed Freeman as Cloud Apprentice of the Year. They both won! We’re all immensely proud of them and exceptionally happy that they have […]

GitHub Actions is GitHub’s new CI/CD platform (currently in open beta, at the time of writing). It is comparable with Microsoft’s other CI/CD offering, Azure Pipelines, which forms part of the Azure DevOps suite. Being fairly well acquainted with Azure Pipelines, I found myself looking for comparisons when getting started with GitHub Actions. It became […]

Azure Functions are powerful and convenient extension points for your Azure Data Factory pipelines. Put your custom processing logic behind an HTTP triggered Azure Function and you are good to go. Unfortunately many people read the Azure documentation and assume they can merrily run a Function for up to 10 minutes on a consumption plan […]

So, the title may be somewhat misleading… But I’ve got you here now… For those who don’t know, endjin is a fully remote company, this is brilliant in a lot of ways (which I won’t list now) but one of the main bonuses is it gives us the freedom to work from wherever we happen […]

Last week, everyone at endjin was presented with a Raspberry Pi 4, with the intention that we would go away and find cool stuff to do with it. My colleague, Jon, has already written up a post describing how to run Azure Functions in a Docker container on the Pi. One of the best ways […]

Over the past few months I’ve worked on multiple projects involving reactive processing of large amounts of data. In a world where the volume of data is increasing at an almost inconceivable rate, being able to process this data efficiently and cheaply is vital. A large part of the work involved in these projects was […]

At our endjin team meet up this week, we were all presented with Raspberry Pi 4b’s and told to go away and think of something good to do with them. I first bought a Raspberry Pi back in 2012 and have to admit, beyond installing XBMC and playing around with it, I haven’t done a […]

Sometimes we need to import and export notebooks from a Databricks workspace. This might be because you have a bunch of generic notebooks that can be useful across numerous workspaces, or it could be that you’re having to delete your current workspace for some reason and therefore need to transfer content over to a new […]

TL;DR I wanted a nice way of animating in a D3 donut chart with a “corkscrew” effect, where segments animate in at constant speed regardless of segment size in a clockwise sweep, simultaneously animating out from the centre. Here’s what I came up with: See the Pen D3.js pie/donut chart corkscrew entry animation by Jonathan […]

A couple of weeks ago I had the opportunity to attend NDC in Oslo. It was an absolutely brilliant experience, and my head is still reeling a bit from everything I learnt! The focus of a lot of the talks was around neural networks and machine learning – something which we have explored quite a […]

Have you been trying to create a Databricks cluster using the CLI? Have you been getting infuriated by something seemingly so trivial? Well, join the club. Although, get ready to depart the club because I may have the solution you need. When creating a cluster using the CLI command databricks clusters create, you’re required to […]

So, this week I’m moving on from the creational patterns (though I might return at some point) and moving on to the structural design patterns! Structural design patterns are ones that deal with the relationships between different elements. Essentially, they define how different parts of the code interact with and relate to each other. This […]

Here at endjin we spend a lot of time working with data, and securing that data is top on our list of priorities . Therefore, anything we can do to reduce the need for storing access keys is a huge win! (Here is a guest blog from Barry Smart at Hymans Robertson which details our […]

So, the other week I took a look at the Factory Method and Abstract Factory design patterns. This week I’m continuing with the creational design patterns, this time focusing on the builder pattern! The builder pattern is used when there is complex set up involved in creating an object. Like the other creational patterns, it […]

Whilst some of the Azure Active Directory PowerShell for Graph module (AzureAD) functionality has been rolled into the new Azure PowerShell Az module (Az), it’s not currently (and might never be) a replacement for the full power of what you can achieve with AzureAD. So, there’s every chance you’ll find yourself needing to use both […]

I’ve just completed my first year working full time for endjin, after I joined as an Apprentice II last May. And without sounding too much like a cliché, time has truly flown. In the ins-and-outs of daily working life I haven’t really noticed the huge progress I’ve made in the last year, but as we […]

I’ve been focusing lately on code quality, refactoring and architecture. A large part of my reading (& Pluralsight course-ing) has been based around design patterns, and how the application of these patterns can help bring your code in line with the SOLID principles. This means that code which uses these patterns is often more extensible, […]

As part of a recent project, we needed to automate the creation of a Power BI workspace. This workspace was part of a bigger deployment that provisioned Azure resources (including a Power BI Embedded capacity), and was orchestrated via an Azure DevOps Pipeline. Whilst there’s a tonne of useful information out there, there was no […]

I recently wrote a blog on using ADF Mapping Data Flow for data manipulation. As part of the same project, we also ported some of an existing ETL Jupyter notebook, written using the Python Pandas library, into a Databricks Notebook. This notebook could then be run as an activity in a ADF pipeline, and combined […]

I’ve very pleased to announce that endjin has become a Snowflake partner. This fantastic “designed for the cloud” data platform redefines what a data warehouse can be in the age of cloud. With features such as data sharing, usage based billing, and availability on Microsoft Azure, it has won our hearts. Over the last three years, we’ve […]