Endjin - Home

Machine Learning

NDC London 2020 – My highlights

by Ed Freeman

A couple of weeks back, along with a rabble of other endjineers, I was fortunate enough to attend NDC London. This wasn’t my first time at an NDC conference – in fact, my previous outing was to Oslo to experience the “original” flavour of NDC back in 2018. That was extremely fun and packed with […]


AI for Good Hackathon

by Ian Griffiths

Towards the end of last year, Microsoft invited endjin along to a hackathon session they hosted at the IET in London as part of their AI for Good initiative. I’ve been thinking about the event and the broader work Microsoft is doing here a lot lately, because it gets to the heart of what I love about working in this industry: computers can magnify our power to do to good.


Very excited to be speaking at NDC in London in January! The talk is focused on “Combatting illegal fishing with Machine Learning and Azure” and will focus on the recent work we did with OceanMind. OceanMind are a not-for-profit who are working on cleaning up the world’s oceans with the help of Microsoft’s cloud technologies. […]


Import and export notebooks in Databricks

by Ed Freeman

Sometimes we need to import and export notebooks from a Databricks workspace. This might be because you have a bunch of generic notebooks that can be useful across numerous workspaces, or it could be that you’re having to delete your current workspace for some reason and therefore need to transfer content over to a new […]


Machine learning often seems like a black box. This post walks through what’s actually happening under the covers, in an attempt to de-mystify the process!

Neural networks are built up of neurons. In a shallow neural network we have an input layer, a “hidden” layer of neurons, and an output layer. For deep learning, there is simply more hidden layers which allows for combining neuron’s inputs and outputs to build up a more detailed picture.

If you have an interest in Machine Learning and what is really happening, definitely give this a read (WARNING: Some algebra ahead…)!


Here at endjin we’ve done a lot of work around data analysis and ETL. As part of this we have done some work with Databricks Notebooks on Microsoft Azure. Notebooks can be used for complex and powerful data analysis using Spark. Spark is a “unified analytics engine for big data and machine learning”. It allows you to run data analysis workloads, and can be accessed via many APIs. This means that you can build up data processes and models using a language you feel comfortable with. They can also be run as an activity in a ADF pipeline, and combined with Mapping Data Flows to build up a complex ETL process which can be run via ADF.


ML.NET, Azure Functions and the 4th Industrial Revolution

by Howard van Rooijen

TLDR; There is a lot of hype around AI & ML. Here’s an example of using ML.NET & Azure Functions to deliver a series of micro-optimisations, to automate a series of 1 second tasks. When applied to business processes, this is what the 4th Industrial Revolution could look like. We’re in the 3rd major hype […]


When he joined endjin, Technical Fellow Ian sat down with founder Howard for a Q&A session. This was originally published on LinkedIn in 5 parts, but is republished here, in full. Ian talks about his path into computing, some highlights of his career, the evolution of the .NET ecosystem, AI, and the software engineering life.


Using Python inside SQL Server

by Ed Freeman

Hello everyone. Before Christmas I played around with SQL Server 2017’s inline Python integration capability. This capability was announced early last year, with the corresponding integration with R already being possible for a number of months. The main benefits from this are the abilities to: Eliminate data movement (having to transfer data samples from a database to […]


My first month as an apprentice at endjin

by Ed Freeman

A little over 10 months ago I wrote a blog which reflected on my summer internship at endjin. Now, a little over 10 months later, I’m sitting in the same office one month into my Software Engineer Apprenticeship after having completed my Maths degree and choosing to come and join the team here, for good. […]


2 Day Microsoft Bot Framework Hackathon with Watchfinder

by Howard van Rooijen

Jonathan Gill CTO, Watchfinder Jonathan Gill is CTO at Watchfinder. Watchfinder buys watches from members of the public, returns them to as new condition, warranties them, and sells them back to the public via their website and retail stores. They have grown to 120+ staff, and turnover £70 million per year. On the 3rd of […]


Guest Blogger – Ed describes his three month paid summer internship at endjin Ed is studying a Mathematics BSc at UCL. He made contact in January because he was looking to gain some real world software development experience during his summer break, before his final year at University. During his three month internship we tried to expose him to as many different […]


Choosing the right cloud platform provider can be a daunting task. Take the big three, AWS, Azure, and Google Cloud Platform; each offer a huge number of products and services, but understanding how they enable your specific needs is not easy. Since most organisations plan to migrate existing applications it is important to understand how […]


Automating R Unit Tests With VSTS

by Jess Panni

I recently demonstrated how it was possible to automate the deployment of R models to Azure Machine Learning through VSTS. Of course, this is only part of the story; what about testing? It is important to ensure that all production code is adequately tested, and R is no different. Writing unit tests for R models is straight-forward […]


What is Azure Machine Learning? Azure Machine Learning (Azure ML) is a fully managed cloud service that enables you to easily build, deploy and share predictive analytics solutions. Azure ML allows you to create a predictive analytic experiment and then directly publish that as a web service. The web service API can be used in […]


Automated R Deployments in Azure

by Jess Panni

It’s been great to see Microsoft embracing the R language on Azure, being able to easily operationalize R assets is changing the way organisations think about their analytical workloads. While it is trivial to publish an R model as a web service in Azure Machine Learning, there is still no easy way to integrate this […]


Machine Learning – the process is the science

by James Broome

What do machine learning and data science actually mean? This post digs into the detail behind the endjin approach to structured experimentation, arguing that the “science” is really all about following the process, allowing you to iterate to insights quickly when there are no guarantees of success.


We produced a booklet to coincide with our Future Decoded talk “The 100 Year Start-up: Embracing Disruption in Financial Services“, where we examine the challenges and opportunities in the Microsoft Cloud for the Financial Services Industry, covering the following topics: Security, Privacy & Data Sovereignty Data Ingestion, Transformation & Enrichment Big Compute Big Data – […]


This post looks at what machine learning really is (and isn’t), dispelling some of the myths and hype that have emerged as the interest in data science, predictive analytics and machine learning has grown. Without any hard guarantees of success, it argues that machine learning as a discipline is simply trial and error at scale – proving or disproving statistical scenarios through structured experimentation.


A short while ago, I was trying to classify some data using Azure Machine Learning, but the training data was very imbalanced. In the attempt to build a useful model from this data, I came across the Synthetic Minority Oversampling Technique (SMOTE), an approach to dealing with imbalanced training data. This blog describes what I […]