Endjin - Home

Machine Learning

The theme of this year’s British Science Week (6 – 15 March 2020) is “Our Diverse Planet”. We’ll be getting involved by speaking to school children about the work we’ve been doing with Oxfordshire-based OceanMind (part of the Microsoft AI for Good programme) to help them combat illegal fishing, hopefully inspiring some of the next generation of data scientists!

NDC London 2020 – My highlights

by Ed Freeman

A couple of weeks back, along with a rabble of other endjineers, I was fortunate enough to attend NDC London. This wasn’t my first time at an NDC conference – in fact, my previous outing was to Oslo to experience the “original” flavour of NDC back in 2018. That was extremely fun and packed with […]

AI for Good Hackathon

by Ian Griffiths

Towards the end of last year, Microsoft invited endjin along to a hackathon session they hosted at the IET in London as part of their AI for Good initiative. I’ve been thinking about the event and the broader work Microsoft is doing here a lot lately, because it gets to the heart of what I love about working in this industry: computers can magnify our power to do to good.

Very excited to be speaking at NDC in London in January! The talk is focused on “Combatting illegal fishing with Machine Learning and Azure” and will focus on the recent work we did with OceanMind. OceanMind are a not-for-profit who are working on cleaning up the world’s oceans with the help of Microsoft’s cloud technologies. […]

Import and export notebooks in Databricks

by Ed Freeman

Sometimes it’s necessary to import and export notebooks from a Databricks workspace. This might be because you have some generic notebooks that can be useful across numerous workspaces, or it could be that you’re having to delete your current workspace for some reason and therefore need to transfer content over to a new workspace. Importing and exporting can be doing either manually or programmatically. In this blog, we outline a way to recursively export/import a directory and its files from/to a Databricks workspace.

Machine learning often seems like a black box. This post walks through what’s actually happening under the covers, in an attempt to de-mystify the process!

Neural networks are built up of neurons. In a shallow neural network we have an input layer, a “hidden” layer of neurons, and an output layer. For deep learning, there is simply more hidden layers which allows for combining neuron’s inputs and outputs to build up a more detailed picture.

If you have an interest in Machine Learning and what is really happening, definitely give this a read (WARNING: Some algebra ahead…)!

Here at endjin we’ve done a lot of work around data analysis and ETL. As part of this we have done some work with Databricks Notebooks on Microsoft Azure. Notebooks can be used for complex and powerful data analysis using Spark. Spark is a “unified analytics engine for big data and machine learning”. It allows you to run data analysis workloads, and can be accessed via many APIs. This means that you can build up data processes and models using a language you feel comfortable with. They can also be run as an activity in a ADF pipeline, and combined with Mapping Data Flows to build up a complex ETL process which can be run via ADF.

ML.NET, Azure Functions and the 4th Industrial Revolution

by Howard van Rooijen

TLDR; There is a lot of hype around AI & ML. Here’s an example of using ML.NET & Azure Functions to deliver a series of micro-optimisations, to automate a series of 1 second tasks. When applied to business processes, this is what the 4th Industrial Revolution could look like. We’re in the 3rd major hype […]

When he joined endjin, Technical Fellow Ian sat down with founder Howard for a Q&A session. This was originally published on LinkedIn in 5 parts, but is republished here, in full. Ian talks about his path into computing, some highlights of his career, the evolution of the .NET ecosystem, AI, and the software engineering life.

Using Python inside SQL Server

by Ed Freeman

Do you have a bunch of data in SQL Server that you’re using ODBC/JDBC to pull data to work with in Python? Using SQL Server’s Python integration, you can connect to a SQL Server instance within your preferred IDE and perform the computations on the SQL Server Machine. No more clunky data transferring. Operationalizing a Python model/script is as easy as calling a stored procedure. Any application that can speak to SQL Server can invoke the Python code and retrieve the results. Easy! This blog will provide a few, simple examples which make use of this capability to carry out some simple Python commands, so you can get up and running as quickly as possible.

Learn what types of things an apprentice gets up to at endjin a few months after joining. You could be learning about Neural Networks: algorithms which mimic the way biological systems process information. You could be attending Microsoft’s Future Decoded conference, learning about Bots, CosmosDB, IoT and much more. Hopefully, you wouldn’t be in hospital after a ruptured appendix!

My first month as an apprentice at endjin

by Ed Freeman

Structured apprenticeships provide a great way to build skills whilst getting real-life experience. Endjin’s apprenticeship scheme has been refined over years, with an optimal mixture of training, project work, and exposure to commercial processes – a scheme which is designed to build strong foundations for a well-rounded Software Engineering consultant. This post explains the transition from university to an apprenticeship at endjin, including the types of work an apprentice could end up doing, and some examples of real-life learnings from a real-life apprentice.

2 Day Microsoft Bot Framework Hackathon with Watchfinder

by Howard van Rooijen

Jonathan Gill CTO, Watchfinder Jonathan Gill is CTO at Watchfinder. Watchfinder buys watches from members of the public, returns them to as new condition, warranties them, and sells them back to the public via their website and retail stores. They have grown to 120+ staff, and turnover £70 million per year. On the 3rd of […]

A career in software engineering doesn’t need to start with a Computer Science degree. The underlying traits of problem solving, a willingness to learn and the ability to collaborate well can be built in any field. Internships provide a great way to get your foot-in-the-door in the professional world, and arm you with some real-life experience for future endeavours. This post describes an internship at endjin, including the type of work you could be asked to do and what you could learn.

Choosing the right cloud platform provider can be a daunting task. Take the big three, AWS, Azure, and Google Cloud Platform; each offer a huge number of products and services, but understanding how they enable your specific needs is not easy. Since most organisations plan to migrate existing applications it is important to understand how […]

Automating R Unit Tests With VSTS

by Jess Panni

I recently demonstrated how it was possible to automate the deployment of R models to Azure Machine Learning through VSTS. Of course, this is only part of the story; what about testing? It is important to ensure that all production code is adequately tested, and R is no different. Writing unit tests for R models is straight-forward […]

What is Azure Machine Learning? Azure Machine Learning (Azure ML) is a fully managed cloud service that enables you to easily build, deploy and share predictive analytics solutions. Azure ML allows you to create a predictive analytic experiment and then directly publish that as a web service. The web service API can be used in […]

Automated R Deployments in Azure

by Jess Panni

It’s been great to see Microsoft embracing the R language on Azure, being able to easily operationalize R assets is changing the way organisations think about their analytical workloads. While it is trivial to publish an R model as a web service in Azure Machine Learning, there is still no easy way to integrate this […]

Machine Learning – the process is the science

by James Broome

What do machine learning and data science actually mean? This post digs into the detail behind the endjin approach to structured experimentation, arguing that the “science” is really all about following the process, allowing you to iterate to insights quickly when there are no guarantees of success.

We produced a booklet to coincide with our Future Decoded talk “The 100 Year Start-up: Embracing Disruption in Financial Services“, where we examine the challenges and opportunities in the Microsoft Cloud for the Financial Services Industry, covering the following topics: Security, Privacy & Data Sovereignty Data Ingestion, Transformation & Enrichment Big Compute Big Data – […]