Platform Engineering

We regularly write about our technical experiences (good and bad) and what we're learning from the market.

Tags

Knocking on Azure's Front Door

Posted by Ross McDermott on 18 April 2019

Azure, Azure Front Door, Networking

The ability to create global scale web applications using Azure infrastructure and services has existed for quite some time, but has traditionally been complex to setup and manage with creation of both global, and region resources. With the go-live release of the Azure Front Door (AFD) service, this just got a whole...

Continue reading

Dashboarding JVM metrics from OpenShift pods

Posted by James Garner on 16 April 2019

docker, kubernetes, openshift, grafana, dashboard, influxdb, jolokia, jmx

Recently on a client engagement, I needed to extract some real-time metrics from some Mule pods running in an OpenShift environment. At that point in time, we were engaged in a performance testing activity, and while the tests were actually passing successfully, the lack of visibility into the pods made me nervous. I...

Continue reading

Automate Kafka Testing

Posted by Tabish Ghani on 12 April 2019

Testing, unit-testing, kafka, Fast-Data

Overview

Apache Kafka is being leveraged very commonly and forms some of large scale and important systems in the world processing trillions of messages per day. It is serving as pipeline backbone for many companies in financial and tech industry.

Before I continue, I want to set some expectations. The point of this...

Continue reading

Enterprise Integration Patterns on Azure Integration Services - Message Construction

Posted by Paco de la Cruz on 10 April 2019

Azure Functions, Enterprise Integration Patterns, Logic Apps, Service Bus, Azure Event Grid, Azure iPaaS

When we are designing a message-based integration solution, one of the first things we need to define is how the application data are going to be serialised into messages, so they can be transmitted to other applications. The Message Construction Enterprise Integration Patterns provide documented knowledge that can...

Continue reading

Back to the future

Posted by Sarah Fernando on 05 April 2019

work life balance, career break, return to work, people & culture

A story of a mom who believed she would never return to work

I joined Deloitte Platform Engineering (DPE) in November 2018. This is my first job in sunny Australia. Having moved to Melbourne from London almost five years back, returning to work has been a daunting idea which turned out to be a surprisingly pleasant...

Continue reading

Evolution of the Solution Architect Role

Posted by Graeme Perrins on 04 April 2019

Solution Architecture, Solution Architects

Defining Solution Architecture 

Having worked in the design and creation of software based solutions over many years, I have seen firsthand the need for the solution architect to continually learn and adapt to the evolution of architecture design patterns, technologies and methods used to deliver solutions.

To the...

Continue reading

Write and deploy an Apache Beam pipeline with Dataflow

Posted by Sheng Wu on 02 April 2019

tech, gcp, dataflow, Apache Beam, Fast-Data, parquet, csv

Overview 

Apache Beam is a unified programming model and the name Beam means Batch + strEAM. It is good at processing both batch and streaming data and can be run on different runners, such as Google Dataflow, Apache Spark, and Apache Flink. The Beam programming guide documents on how to develop a pipeline and the ...

Continue reading

Access KSQL server in Google Kubernetes Engine locally in 5 steps

Posted by Tabish Ghani on 25 March 2019

kubernetes, container, kafka, dataflow, Fast-Data

Overview

Apache Kafka allows both local and cloud deployment so you can publish data from on premise environment and trigger services in the cloud. It is at the heart of our stacks that require real time processing. Confluent KSQL (streaming engine) allows stream processing in a simple and interactive SQL interface...

Continue reading

Protecting Data Privacy - Obligations for solution architects

Posted by Graeme Perrins on 22 March 2019

Architecture, data privacy, Solution Architecture, Solution Architects

Data Privacy Failures 

The last couple of years has seen a turning point regarding the increased awareness that people have of how organisations use their data and the responsibilities that come with holding personal data.

 

Recent Examples

Following some significant failings of companies to protect data privacy:

Continue reading

Mule on OpenShift: Part 2 - Build & Deploy

Posted by Sohrab Hosseini on 15 March 2019

docker, kubernetes, tech, mule, platform, openshift, container, anypoint

In part 1 of this series, we discussed the different deployment models that we have used in the past to deploy containerised Mule applications on OpenShift Container Platform. Here we expand on the topic by discussing best practices around build and deployment such applications.

Container Base Image

In a...

Continue reading