Endjin - Home

Big Data

We’re currently building a Data Governance Platform product that enables UK Financial Services organisations to discover and manage the life-cycle, usage, risk and compliance requirements of data assets across the organisation. Much of the core functionality is delivered using Cosmos DB’s Gremlin API to model data lineage and other relationships best represented by a graph […]


Overflowing with dataflow part 1: An overview

by Carmel Eve

In a recent project, I was asked to produce a tool for importing a fairly large amount of data at once, this data then needed to be processed and exported. After much refactoring, I achieved a solution I was satisfied with which used TPL dataflow to execute the processing in parallel. Before I talk specifically […]


Using Python inside SQL Server

by Ed Freeman

Hello everyone. Before Christmas I played around with SQL Server 2017’s inline Python integration capability. This capability was announced early last year, with the corresponding integration with R already being possible for a number of months. The main benefits from this are the abilities to: Eliminate data movement (having to transfer data samples from a database to […]


How to plan your cloud transformation journey

by Howard van Rooijen

This week I received an email from someone who asked how they could use our free Thought Leadership content to help their organisation move to the cloud. I realised that although we’ve released a lot of content, we’d never talked publicly about the rationale behind them and how they are all interconnected. Our Thought Leadership […]


This post explains how to create a PowerBI report which sources data from two separate Azure SQL Databases. PowerBI offers two data access mechanisms; Import and DirectQuery. DirectQuery provides a range of benefits, the chief of which is that data is automatically refreshed. DirectQuery doesn’t import any data into PowerBI, instead it queries the data […]


Choosing the right cloud platform provider can be a daunting task. Take the big three, AWS, Azure, and Google Cloud Platform; each offer a huge number of products and services, but understanding how they enable your specific needs is not easy. Since most organisations plan to migrate existing applications it is important to understand how […]


In this series, we’re comparing cloud services from AWS, Azure and Google Cloud Platform. A full breakdown and comparison of cloud providers and their services are available in this handy poster. We have assessed services across three typical migration strategies: Lift and shift – the cloud service can support running legacy systems with minimal change […]


In February 2016, I completed my second year of endjin’s three year custom apprenticeship scheme. This blog is a chance for me to reflect on what was learnt over the year – hopefully others will find it useful too. Year one had involved a very steep learning curve as I transitioned from student to graduate […]


We produced a booklet to coincide with our Future Decoded talk “The 100 Year Start-up: Embracing Disruption in Financial Services“, where we examine the challenges and opportunities in the Microsoft Cloud for the Financial Services Industry, covering the following topics: Security, Privacy & Data Sovereignty Data Ingestion, Transformation & Enrichment Big Compute Big Data – […]