Last week, everyone at endjin was presented with a Raspberry Pi 4, with the intention that we would go away and find cool stuff to do with it. My colleague, Jon, has already written up a post describing how to run Azure Functions in a Docker container on the Pi. One of the best ways […]

Over the past few months I’ve worked on multiple projects involving reactive processing of large amounts of data. In a world where the volume of data is increasing at an almost inconceivable rate, being able to process this data efficiently and cheaply is vital. A large part of the work involved in these projects was […]

At our endjin team meet up this week, we were all presented with Raspberry Pi 4b’s and told to go away and think of something good to do with them. I first bought a Raspberry Pi back in 2012 and have to admit, beyond installing XBMC and playing around with it, I haven’t done a […]

Sometimes we need to import and export notebooks from a Databricks workspace. This might be because you have a bunch of generic notebooks that can be useful across numerous workspaces, or it could be that you’re having to delete your current workspace for some reason and therefore need to transfer content over to a new […]

TL;DR I wanted a nice way of animating in a D3 donut chart with a “corkscrew” effect, where segments animate in at constant speed regardless of segment size in a clockwise sweep, simultaneously animating out from the centre. Here’s what I came up with: See the Pen D3.js pie/donut chart corkscrew entry animation by Jonathan […]

A couple of weeks ago I had the opportunity to attend NDC in Oslo. It was an absolutely brilliant experience, and my head is still reeling a bit from everything I learnt! The focus of a lot of the talks was around neural networks and machine learning – something which we have explored quite a […]

Have you been trying to create a Databricks cluster using the CLI? Have you been getting infuriated by something seemingly so trivial? Well, join the club. Although, get ready to depart the club because I may have the solution you need. When creating a cluster using the CLI command databricks clusters create, you’re required to […]

So, this week I’m moving on from the creational patterns (though I might return at some point) and moving on to the structural design patterns! Structural design patterns are ones that deal with the relationships between different elements. Essentially, they define how different parts of the code interact with and relate to each other. This […]

Here at endjin we spend a lot of time working with data, and securing that data is top on our list of priorities . Therefore, anything we can do to reduce the need for storing access keys is a huge win! (Here is a guest blog from Barry Smart at Hymans Robertson which details our […]

So, the other week I took a look at the Factory Method and Abstract Factory design patterns. This week I’m continuing with the creational design patterns, this time focusing on the builder pattern! The builder pattern is used when there is complex set up involved in creating an object. Like the other creational patterns, it […]

Whilst some of the Azure Active Directory PowerShell for Graph module (AzureAD) functionality has been rolled into the new Azure PowerShell Az module (Az), it’s not currently (and might never be) a replacement for the full power of what you can achieve with AzureAD. So, there’s every chance you’ll find yourself needing to use both […]

I’ve just completed my first year working full time for endjin, after I joined as an Apprentice II last May. And without sounding too much like a cliché, time has truly flown. In the ins-and-outs of daily working life I haven’t really noticed the huge progress I’ve made in the last year, but as we […]

I’ve been focusing lately on code quality, refactoring and architecture. A large part of my reading (& Pluralsight course-ing) has been based around design patterns, and how the application of these patterns can help bring your code in line with the SOLID principles. This means that code which uses these patterns is often more extensible, […]

As part of a recent project, we needed to automate the creation of a Power BI workspace. This workspace was part of a bigger deployment that provisioned Azure resources (including a Power BI Embedded capacity), and was orchestrated via an Azure DevOps Pipeline. Whilst there’s a tonne of useful information out there, there was no […]

I recently wrote a blog on using ADF Mapping Data Flow for data manipulation. As part of the same project, we also ported some of an existing ETL Jupyter notebook, written using the Python Pandas library, into a Databricks Notebook. This notebook could then be run as an activity in a ADF pipeline, and combined […]

I’ve very pleased to announce that endjin has become a Snowflake partner. This fantastic “designed for the cloud” data platform redefines what a data warehouse can be in the age of cloud. With features such as data sharing, usage based billing, and availability on Microsoft Azure, it has won our hearts. Over the last three years, we’ve […]

Five editions? Already? How time flies. The Power BI Weekly newsletter is proving a great success – we’ve just published the fifth edition, hundreds of people have subscribed and we’ve received lots of kind feedback. I think it’s safe to say that Power BI has become omnipresent in recent times. We use it widely here at endjin […]

Firstly, I should start by calling out the fact that there are different types of architecture. For example, enterprise architecture focuses on strategy rather than code. It is the strategy for how people, processes, technology etc. is utilised across an organisation in order to be effective and efficient. The type of architecture I will be […]

I originally wrote this post in 2007 (now lost in the annals of time), and is one of my favourite posts. I’m very fond of the confluence of ideas and the serendipity of discovering someone amazing from another discipline articulating the same thoughts I was having. During one of the Open Space sessions I attended […]

As part of a recent project we did a lot of experimentation with the new Azure Data Factory feature: Mapping Data Flows. The tool is still in preview, and more functionality is sure to be in the pipeline, but I think it opens up a lot of really exciting possibilities for visualising and building up […]