Platform Engineering

We regularly write about our technical experiences (good and bad) and what we're learning from the market.

    Tags

    Write and deploy an Apache Beam pipeline with Dataflow

    Posted by Sheng Wu on 02 April 2019

    tech, gcp, dataflow, Apache Beam, Fast-Data, parquet, csv

    Overview 

    Apache Beam is a unified programming model and the name Beam means Batch + strEAM. It is good at processing both batch and streaming data and can be run on different runners, such as Google Dataflow, Apache Spark, and Apache Flink. The Beam programming guide documents on how to develop a pipeline and the ...

    Continue reading

    Mule on OpenShift: Part 2 - Build & Deploy

    Posted by Sohrab Hosseini on 15 March 2019

    docker, kubernetes, tech, mule, platform, openshift, container, anypoint

    In part 1 of this series, we discussed the different deployment models that we have used in the past to deploy containerised Mule applications on OpenShift Container Platform. Here we expand on the topic by discussing best practices around build and deployment such applications.

    Container Base Image

    In a...

    Continue reading

    Mule on OpenShift: Part 1 - Deployment Models

    Posted by Sohrab Hosseini on 28 February 2019

    kubernetes, tech, mule, platform, openshift, container, anypoint

    At Deloitte Platform Engineering, we have been building containerised platforms for a few years now. Red Hat OpenShift Container Platform has been our weapon of choice in many of these instances, complementing a solid Kubernetes foundation.

    A significant part of this work also consists of building and deploying...

    Continue reading

    Journey Towards Event-Driven Microservices

    Posted by Sudarshan Sreenivasan on 26 November 2018

    Events, Microservices, tech, kafka, EDA

    Many organizations are adopting a microservices architecture in order to reduce dependencies between system components and allow more frequent release cycles and more flexible scalability. However, unless they have a clear view of the underlying interaction patterns, teams risk building a tightly coupled distributed...

    Continue reading

    Solving Complex Problems With DataWeave

    Posted by Lipika Naik on 16 November 2018

    tech, java, mule, dataweave, groovy

    DataWeave is the primary transformation language in Mule. What is interesting about DataWeave is that it brings together features of XSLT (mapping), SQL (joinBy, splitBy, orderBy, groupBy, distinctBy operators), Streaming, Functional Programming (use of functions in DataWeave code) to make it a power-packed data...

    Continue reading

    Comparing Istio with Netflix Frameworks for Inter-Microservices Communications

    Posted by Elmira Hasanzadeh on 04 September 2018

    Architecture, Microservices, tech, api, istio, netflix, service mesh, gcp

    The technical advancements in every aspect of software development lifecycle make it clear that there are more than one solution to any problem. In this article I examine Istio’s service mesh capabilities to address issues that developers face while creating microservices and compare it with the widely adopted Netflix...

    Continue reading

    AWS Elasticsearch Snapshot Archival for the Brave

    Posted by Rahul Sharma on 22 August 2018

    Architecture, Logging, tech, aws, audit, archive, elasticsearch

    Data retention has become a crucial topic over the past few years, with organisations continuously exploring better ways to define and implement secure processes that cater to their needs. This becomes increasingly difficult when most of our data is stored/generated in the cloud, hence, requiring a flexible and...

    Continue reading

    Microservices using Spring Boot & Swagger - Part 1

    Posted by Bharat Patel on 10 August 2018

    Microservices, tech, api, java, swagger, spring

    Microservices have become popular as a way of eliminating hidden dependencies between software components and allowing fine-grained deployment without dragging along unneccessary context. In this way, microservices promote autonomy for agile development teams and allow an application to evolve more naturally and in...

    Continue reading

    OpenShift 3 Demystified. For Developers.

    Posted by Sohrab Hosseini on 30 October 2017

    tech, developer, platform, openshift, container

    Container orchestrators are the next big thing in the modern containerisation movement. And unless you have painted your career into a legacy corner, you will encounter these technologies sooner or later.

    OpenShift was my introduction to this space. At that time, I was not able to find suitable online resources that...

    Continue reading

    Monitoring Anypoint CloudHub API Gateway using Amazon Lambda and ELK

    Posted by Nakul Bharade on 26 July 2016

    Serverless, tech, elasticsearch, mule, lambda

    In a previous blog post, Rashmi showed us how to synchronise logs from CloudHub to an external logging system. This follows an increasing demand to utilise MuleSoft Anypoint CloudHub logs, events, and dashboard statistics as part of a broader monitoring strategy which aims to:

    • Aggregate information in a central...
    Continue reading