Platform Engineering

We regularly write about our technical experiences (good and bad) and what we're learning from the market.

    Tags

    APIdays Returns to Melbourne

    Posted by Saul Caganoff on 03 July 2019

    APIs, containers, kubernetes, Fast-Data, Open Banking, APIdays, open data, cloud computing, ethics

    APIdays returns to Melbourne on September 19 and 20. This is our seventh event in Australia/New Zealand on top of 40+ APIdays events globally. As we plan for this year and our call for speakers it is worth looking back at the last 12 months in an API connected world.

    Open Banking and Open Data

    Financial services...

    Continue reading

    Automate Kafka Testing

    Posted by Tabish Ghani on 12 April 2019

    Testing, unit-testing, kafka, Fast-Data

    Overview

    Apache Kafka is being leveraged very commonly and forms some of large scale and important systems in the world processing trillions of messages per day. It is serving as pipeline backbone for many companies in financial and tech industry.

    Before I continue, I want to set some expectations. The point of this...

    Continue reading

    Write and deploy an Apache Beam pipeline with Dataflow

    Posted by Sheng Wu on 02 April 2019

    tech, gcp, dataflow, Apache Beam, Fast-Data, parquet, csv

    Overview 

    Apache Beam is a unified programming model and the name Beam means Batch + strEAM. It is good at processing both batch and streaming data and can be run on different runners, such as Google Dataflow, Apache Spark, and Apache Flink. The Beam programming guide documents on how to develop a pipeline and the ...

    Continue reading

    Access KSQL server in Google Kubernetes Engine locally in 5 steps

    Posted by Tabish Ghani on 25 March 2019

    kubernetes, container, kafka, dataflow, Fast-Data

    Overview

    Apache Kafka allows both local and cloud deployment so you can publish data from on premise environment and trigger services in the cloud. It is at the heart of our stacks that require real time processing. Confluent KSQL (streaming engine) allows stream processing in a simple and interactive SQL interface...

    Continue reading