Platform Engineering

We regularly write about our technical experiences (good and bad) and what we're learning from the market.

Tags

Enterprise Integration Patterns on Azure Integration Services – Messaging Channels

Posted by Paco de la Cruz on 09 May 2019

Azure Functions, Enterprise Integration Patterns, Logic Apps, Service Bus, Azure Event Grid, messaging, Azure Integration Services

In the previous post of the series, I covered how application data are to be serialised and packaged into messages so they can be transmitted between applications. In this post, I’ll describe the Messaging Channels patterns, which focus on solving the challenges of transmitting messages from a sender application to...

Continue reading

6 Ways Passing Secrets to ARM Templates

Posted by Justin Yoo on 24 April 2019

Azure Resource Manager, ARM Templates, DevOps, Azure Key Vault, Azure Pipelines, YARM CLI, Azure DevOps, Linked Templates

Whenever you deal with ARM templates, you always face to handle some sensitive information. This is mainly for API keys handling. How can you cope with those values other than hard-code them into the templates? There are six different ways to handle them we’re going to discuss in this post.

1. Use ARM Template...

Continue reading

Knocking on Azure's Front Door

Posted by Ross McDermott on 18 April 2019

Azure, Azure Front Door, Networking

The ability to create global scale web applications using Azure infrastructure and services has existed for quite some time, but has traditionally been complex to setup and manage with creation of both global, and region resources. With the go-live release of the Azure Front Door (AFD) service, this just got a whole...

Continue reading

Dashboarding JVM metrics from OpenShift pods

Posted by James Garner on 16 April 2019

docker, kubernetes, openshift, grafana, dashboard, influxdb, jolokia, jmx

Recently on a client engagement, I needed to extract some real-time metrics from some Mule pods running in an OpenShift environment. At that point in time, we were engaged in a performance testing activity, and while the tests were actually passing successfully, the lack of visibility into the pods made me nervous. I...

Continue reading

Automate Kafka Testing

Posted by Tabish Ghani on 12 April 2019

Testing, unit-testing, kafka, Fast-Data

Overview

Apache Kafka is being leveraged very commonly and forms some of large scale and important systems in the world processing trillions of messages per day. It is serving as pipeline backbone for many companies in financial and tech industry.

Before I continue, I want to set some expectations. The point of this...

Continue reading

Enterprise Integration Patterns on Azure Integration Services - Message Construction

Posted by Paco de la Cruz on 10 April 2019

Azure Functions, Enterprise Integration Patterns, Logic Apps, Service Bus, Azure Event Grid, Azure iPaaS

When we are designing a message-based integration solution, one of the first things we need to define is how the application data are going to be serialised into messages, so they can be transmitted to other applications. The Message Construction Enterprise Integration Patterns provide documented knowledge that can...

Continue reading

Back to the future

Posted by Sarah Fernando on 05 April 2019

work life balance, career break, return to work, people & culture

A story of a mom who believed she would never return to work

I joined Deloitte Platform Engineering (DPE) in November 2018. This is my first job in sunny Australia. Having moved to Melbourne from London almost five years back, returning to work has been a daunting idea which turned out to be a surprisingly pleasant...

Continue reading

Evolution of the Solution Architect Role

Posted by Graeme Perrins on 04 April 2019

Solution Architecture, Solution Architects

Defining Solution Architecture 

Having worked in the design and creation of software based solutions over many years, I have seen firsthand the need for the solution architect to continually learn and adapt to the evolution of architecture design patterns, technologies and methods used to deliver solutions.

To the...

Continue reading

Write and deploy an Apache Beam pipeline with Dataflow

Posted by Sheng Wu on 02 April 2019

tech, gcp, dataflow, Apache Beam, Fast-Data, parquet, csv

Overview 

Apache Beam is a unified programming model and the name Beam means Batch + strEAM. It is good at processing both batch and streaming data and can be run on different runners, such as Google Dataflow, Apache Spark, and Apache Flink. The Beam programming guide documents on how to develop a pipeline and the ...

Continue reading

Access KSQL server in Google Kubernetes Engine locally in 5 steps

Posted by Tabish Ghani on 25 March 2019

kubernetes, container, kafka, dataflow, Fast-Data

Overview

Apache Kafka allows both local and cloud deployment so you can publish data from on premise environment and trigger services in the cloud. It is at the heart of our stacks that require real time processing. Confluent KSQL (streaming engine) allows stream processing in a simple and interactive SQL interface...

Continue reading