Endjin - Home

endjin blogs

Think Smarter

Here at endjin we’ve done a lot of work around data analysis and ETL. As part of this we have done some work with Databricks Notebooks on Microsoft Azure. Notebooks can be used for complex and powerful data analysis using Spark. Spark is a “unified analytics engine for big data and machine learning”. It allows you to run data analysis workloads, and can be accessed via many APIs. This means that you can build up data processes and models using a language you feel comfortable with. They can also be run as an activity in a ADF pipeline, and combined with Mapping Data Flows to build up a complex ETL process which can be run via ADF.

Endjin is a Snowflake Partner

by Howard van Rooijen

I’ve very pleased to announce that endjin has become a Snowflake partner. This fantastic “designed for the cloud” data platform redefines what a data warehouse can be in the age of cloud. With features such as data sharing, usage based billing, and availability on Microsoft Azure, it has won our hearts. Over the last three years, we’ve […]

Five editions? Already? How time flies. The Power BI Weekly newsletter is proving a great success – we’ve just published the fifth edition, hundreds of people have subscribed and we’ve received lots of kind feedback. I think it’s safe to say that Power BI has become omnipresent in recent times. We use it widely here at endjin […]

Category: Business Insights

Software architecture and Agile project planning are often seen to be at odds. However, here at endjin we think that the way in which they intersect solves a lot of the common issues surrounding architecture. The key to successful architecture is constantly keeping the drivers in mind and having a tight communication loop as the architecture is implemented. These concepts are ones which are key to agile project management, and the combination of these two disciplines can be extremely powerful and successful when applied correctly.

Agile Architecture: Painting by Numbers

by Howard van Rooijen

I originally wrote this post in 2007 (now lost in the annals of time), and is one of my favourite posts. I’m very fond of the confluence of ideas and the serendipity of discovering someone amazing from another discipline articulating the same thoughts I was having. During one of the Open Space sessions I attended […]

Mapping Data Flows are a relatively new feature of ADF. They allow you to visually build up complex data transformation sequences. This can aid in the streamlining of data manipulation and ETL processes, without the need to write any code! This post gives a brief introduction to the technology, and what this could enable!

In the last post I explained how to create a set of Azure Functions that could load data into Snowflake as well as execute Snowflake queries and export the results into your favorite cloud storage solution. In this post I will show how we can use these functions in Azure Data Factory to plug Snowflake […]

If, like me, you are a fan of Azure Data Factory and love Snowflake then you are probably disappointed that there isn’t a native Data Factory connector for Snowflake. While we wait for an official connector from Microsoft we have no alternative but to roll our own. In this blog post I will walk you through […]

Announcing Power BI Weekly!

by Howard van Rooijen

We launched Azure Weekly, a free weekly newsletter, back in 2014. 200+ issues and many thousands of global subscribers later, it’s still going strong. Last month Ed Freeman pointed out that analytics section of Azure Weekly was mainly full of Power BI articles and that it was by far the largest category in the newsletter. […]

Category: Analytics

At endjin we have a high quality bar when it comes to our code. As part of this we carry out regular code reviews. One of the tools we have used for these code reviews is NDepend. This is the second in a blog series written as we carried out that process. This post focuses on the insight you can quickly gain just by glancing at the NDepend UI.

Enforce resource tagging with Azure Policy

by Mike Larah

This blog post details how we used Azure Policy to enforce Azure resources were tagged with appropiate tags and ensured tags were inherited from parent resource groups where possible.

At endjin we have a high quality bar when it comes to our code. As part of this we carry out regular code reviews. One of the tools we have used for these code reviews is NDepend. This is the first in a blog series written as we carried out that process. This post runs through the different metrics used by NDepend, and the reasons that each of these can be an indication of code quality.

C# faux amis 3: variable declarations and type patterns

by Ian Griffiths

In this, the final article in a series on the potentially deceptive nature of some new features in C# 8, Ian Griffiths shows how using var can sometimes result in better compile-time type checking than using an explicit type. He reflects on the evolution of C#, and what we can learn from this.

Category: Development

C# v8.0 introduces various new patterns. In this article, part of a series on how the evolution of the language has added complexity, Ian Griffiths shows how the strong resemblance between the new Positional Patterns and Deconstruction can be misleading.

In this post Carmel runs through some of the main principles behind agile estimation and planning. At endjin we use a lot of these techniques in our projects and this is a great post which highlights the reasons behind some of what we do. The key motivation behind good estimation is to be useful for project planning. There is a huge amount of inherent uncertainty surrounding estimates, especially early in the project. So, we shift our aim from 100% precise, or “true”, estimates, and towards providing estimates which are useful and accurate.

Carmel also runs through the steps in an agile delivery and release process. Definitely worth the read if you have an interest in agile and/or project management!

C# faux amis 1: discards and underscores

by Ian Griffiths

Visual Studio 2019 saw the arrival of C# v8.0. This article shows the first of a series of examples of how friction can arise when integrating new features into a mature language: the relatively late addition of ‘discards’ causes some surprises.

Category: Development

C# 8 surprising patterns

by Ian Griffiths

Visual Studio 2019 saw the arrival of C# v8.0. A programming language cannot evolve for 20 years without developing a few quirks. Ian Griffiths writes about a surprising aspect of the new pattern matching features, and what this reveals about how the language has changed.

11 cheers for binary (And 3 for hexadecimal)!

by Carmel Eve

Sometimes it’s good to go back to the basics… This is a quick post that runs through binary and hexadecimal numbers, and how those relate to our every day computing!

Category: Technology

This is the final blog in a series which delves into how the Rx operators work under the covers. This series aims to provide a greater understanding of Rx and its operators. This post focuses on the JOIN operator.

This post walks through the fix for DLL locking errors when trying to deploy an Azure Function. The solution was to switch over to the new “deploy from package” option when deploying the functions. This fixes the file locking problem because instead of deploying the DLLs, the function will run from a package file added to its directory.