Platform Engineering

We regularly write about our technical experiences (good and bad) and what we're learning from the market.

Tags

Write and deploy an Apache Beam pipeline with Dataflow

Posted by Sheng Wu on 02 April 2019

tech, gcp, dataflow, Apache Beam, Fast-Data, parquet, csv

Overview 

Apache Beam is a unified programming model and the name Beam means Batch + strEAM. It is good at processing both batch and streaming data and can be run on different runners, such as Google Dataflow, Apache Spark, and Apache Flink. The Beam programming guide documents on how to develop a pipeline and the ...

Continue reading

Access KSQL server in Google Kubernetes Engine locally in 5 steps

Posted by Tabish Ghani on 25 March 2019

kubernetes, container, kafka, dataflow, Fast-Data

Overview

Apache Kafka allows both local and cloud deployment so you can publish data from on premise environment and trigger services in the cloud. It is at the heart of our stacks that require real time processing. Confluent KSQL (streaming engine) allows stream processing in a simple and interactive SQL interface...

Continue reading

Protecting Data Privacy - Obligations for solution architects

Posted by Graeme Perrins on 22 March 2019

Architecture, data privacy, Solution Architecture, Solution Architects

Data Privacy Failures 

The last couple of years has seen a turning point regarding the increased awareness that people have of how organisations use their data and the responsibilities that come with holding personal data.

 

Recent Examples

Following some significant failings of companies to protect data privacy:

Continue reading

Mule on OpenShift: Part 2 - Build & Deploy

Posted by Sohrab Hosseini on 15 March 2019

docker, kubernetes, tech, mule, platform, openshift, container, anypoint

In part 1 of this series, we discussed the different deployment models that we have used in the past to deploy containerised Mule applications on OpenShift Container Platform. Here we expand on the topic by discussing best practices around build and deployment such applications.

Container Base Image

In a...

Continue reading

An Open Letter on Opening Up

Posted by Alan King on 12 March 2019

platforms, mental health, stress, wellbeing, work life balance, feedback, soft skills, support

Introduction

My colleagues here in Platform Engineering have written some excellent blog posts previously in regards to technologies, methodologies and best practices. When I was thinking of a topic to address in this blog post, I decided to talk about something that is finally starting to get exposure and become less...

Continue reading

Enterprise Integration Patterns on Azure Integration Services - Introduction

Posted by Paco de la Cruz on 08 March 2019

Architecture, Enterprise Integration Patterns, Azure Integration Services, Azure iPaaS

In the past, architecting and building integration solutions was a task that only specialised developers and architects were able to do. The tools and skills were considered niche and not every developer was able to build a manageable integration solution. Now, with the available new low-code integration platforms,...

Continue reading

Swift Service Fabric Deployments with PowerShell

Posted by Dan Toomey on 05 March 2019

Azure, Azure Service Fabric, Development, Azure PowerShell, DevOps, Microsoft Azure

A few weeks ago I had the great privilege of presenting a 60 minute breakout session at Microsoft Ignite | The Tour in Sydney. It was thrilling to have over 200 people registered to see my topic “Seamless Deployments with Azure Service Fabric”, especially in the massive Convention Centre. 

In the session I...

Continue reading

Mule on OpenShift: Part 1 - Deployment Models

Posted by Sohrab Hosseini on 28 February 2019

kubernetes, tech, mule, platform, openshift, container, anypoint

At Deloitte Platform Engineering, we have been building containerised platforms for a few years now. Red Hat OpenShift Container Platform has been our weapon of choice in many of these instances, complementing a solid Kubernetes foundation.

A significant part of this work also consists of building and deploying...

Continue reading

Performing Constructor Injections on Azure Functions V2

Posted by Justin Yoo on 22 February 2019

Azure Functions, Dependency Injection, Constructor Injection, Testability

In January 2019, Azure Functions Team has released a new version of its runtime, 2.0.12265.

I wasn’t able to believe what that meant at the first place.

Does that mean we now can get rid of the infamous static modifier from both classes and methods?

In fact, Fabio from Azure Functions Team showed a demo at Ignite...

Continue reading

Journey Towards Event-Driven Microservices

Posted by Sudarshan Sreenivasan on 26 November 2018

Events, Microservices, tech, kafka, EDA

Many organizations are adopting a microservices architecture in order to reduce dependencies between system components and allow more frequent release cycles and more flexible scalability. However, unless they have a clear view of the underlying interaction patterns, teams risk building a tightly coupled distributed...

Continue reading