Endjin - Home

endjin blogs

We help small teams achieve big things.

AI for Good Hackathon

by Ian Griffiths

Towards the end of last year, Microsoft invited endjin along to a hackathon session they hosted at the IET in London as part of their AI for Good initiative. I’ve been thinking about the event and the broader work Microsoft is doing here a lot lately, because it gets to the heart of what I love about working in this industry: computers can magnify our power to do to good.


There are many different paths into the tech industry, Carmel has been speaking at some local schools about joining the industry from a scientific background. In this post she discusses the crucial tools which science gives you which can help you succeed in tech!

Category: Culture

In this blog from the Azure Advent Calendar 2019 we discuss building a secure data solution using Azure Data Lake. Data Lake has many features which enable fine grained security and data separation. It is also built on Azure Storage which enables us to take advantage of all of those features and means that ADLS is still a cost effective storage option!

This post runs through some of the great features of ADLS and runs through an example of how we build our solutions using this technology!


Very excited to be speaking at NDC in London in January! The talk is focused on “Combatting illegal fishing with Machine Learning and Azure” and will focus on the recent work we did with OceanMind. OceanMind are a not-for-profit who are working on cleaning up the world’s oceans with the help of Microsoft’s cloud technologies. […]


Are you configuring your Azure App Service to use a VNet? The regional VNet integration for an Azure App Service is a preview feature, and so comes with some quirks. One of the documented limitations of this preview feature is that “The feature is only available from newer App Service scale units that support PremiumV2 App Service plans.” This has interesting implications as to how you need to deploy your App Service, otherwise you could end up with a rather perplexing pattern of errors. Read about how we came about this error, and how we ended up fixing it.

Category: Cloud, Networking

C# 8 Positional Patterns Custom Deconstructor Pitfall

by Ian Griffiths

The ‘positional patterns’ added in C# 8 support types with custom desconstructors. However, the way this works might always be quite what you would expect. This article shows a surprising behaviour, and explains how it arises.


Survey: Expression Trees use cases in .NET

by Howard van Rooijen

I’m of the opinion of that Expression Trees are one of the most powerful, yet under-used features of the .NET Framework. I’m currently doing research for a bit of thought leadership about how Expression Trees can be used and I’m looking for feedback from the community. If you currently are, or have used Expression Trees […]


C#, Span and async

by Ian Griffiths

The addition of ref struct types, most notably Span, opened C# to a range of high performance scenarios that were impractical to tackle with earlier versions of the language. However, they introduce some challenges. For example, they do not mix very well with async methods. This article shows some techniques for mitigating this.


It’s been an exciting week at endjin! We attended the Computing Rising Star Awards 2019 as both of our current Apprentices were shortlisted; Carmel Eve for Apprentice Engineer of the Year and Ed Freeman as Cloud Apprentice of the Year. They both won! We’re all immensely proud of them and exceptionally happy that they have […]

Category: Culture

GitHub Actions is GitHub’s new CI/CD platform. It is comparable with Azure Pipelines, which forms part of the Azure DevOps suite. In this post, Mike Larah looks at the similarities and differences in the high-level concepts and terminology between the two platforms.


Long Running Functions in Azure Data Factory

by Jess Panni

Azure Functions are powerful and convenient extension points for your Azure Data Factory pipelines. Put your custom processing logic behind an HTTP triggered Azure Function and you are good to go. Unfortunately many people read the Azure documentation and assume they can merrily run a Function for up to 10 minutes on a consumption plan […]


How Azure DevTestLabs is helping me climb Everest

by Carmel Eve

Remote working allows us to work from anywhere we want. This brings a huge amount of flexibility in freedom, however we do need the help of a working laptop! When Carmel’s laptop gave in just before a trip, she used Azure DevTestLabs to allow her to continue to work using a 10 year old Mac that probably couldn’t wouldn’t have been up to the task alone…


Passwordless SSH from Windows 10 to Raspberry Pi

by Mike Larah

In this post, Mike Larah explains how to enable SSH on the Pi, how to access the Pi via SSH using a Windows 10 development machine, and how to enable passwordless SSH access to the Pi.


We worked on a project recently which required us to build a highly performant system for processing vast quantities of messages in real time. We had made the decision to run this processing using Azure Functions with C#. This post runs through some of the techniques we used for writing highly performant, low allocation code, including data streaming, list preallocation and the relatively new C# feature: Span.


Running Azure functions in Docker on a Raspberry Pi 4

by Jonathan George

For one of my first experiments with the Raspberry Pi 4, I decided to get an Azure Function running in a Docker container. This post gives a step-by-step guide on how to do it, as well as providing code you can use a starting point for your own experiments.


Import and export notebooks in Databricks

by Ed Freeman

Sometimes it’s necessary to import and export notebooks from a Databricks workspace. This might be because you have some generic notebooks that can be useful across numerous workspaces, or it could be that you’re having to delete your current workspace for some reason and therefore need to transfer content over to a new workspace. Importing and exporting can be doing either manually or programmatically. In this blog, we outline a way to recursively export/import a directory and its files from/to a Databricks workspace.


Donut chart corkscrew entry animation with d3.js

by Jonathan George

In this post we show a smooth entry animation for D3.js donut (or pie) charts, using a combination of a sweep and a zoom to give a corkscrew-like effect.


Machine learning often seems like a black box. This post walks through what’s actually happening under the covers, in an attempt to de-mystify the process!

Neural networks are built up of neurons. In a shallow neural network we have an input layer, a “hidden” layer of neurons, and an output layer. For deep learning, there is simply more hidden layers which allows for combining neuron’s inputs and outputs to build up a more detailed picture.

If you have an interest in Machine Learning and what is really happening, definitely give this a read (WARNING: Some algebra ahead…)!


Quite often it’s beneficial to work with pre-built CLIs/SDKs to interact with your favourite tools, instead of making requests to the underlying REST API. Much of the complexity around constructing requests has been abstracted, and authentication is often easier. The Databricks CLI makes it easier to interact with your Databricks instance, but sometimes you can run into strange errors when constructing the values passed in as arguments. In this blog, we take a look at a JsonDecodeError that can occur when speaking to the Clusters CLI, and look at a way we can avoid this error.

Category: Big Data

This blog is part of a series around design patterns. This post focuses on the composite pattern. The composite pattern is often used in situations where you want to me able to treat groups and individuals in the same way during processing.