Introduction to Apache Flink by Friedman, Ellen -- Read -- Imperial Library of Trantor

Index

Preface

How to Use This Book Conventions Used in This Book

1. Why Apache Flink?

Consequences of Not Doing Streaming Well

Retail and Marketing The Internet of Things Telecom Banking and Financial Sector

Goals for Processing Continuous Event Data Evolution of Stream Processing Technologies First Look at Apache Flink

Batch and Stream Processing

Flink in Production

Bouygues Telecom Other Examples of Apache Flink in Production

King.com Zalando Otto Group ResearchGate Alibaba Group

Where Flink Fits

2. Stream-First Architecture

Traditional Architecture versus Streaming Architecture Message Transport and Message Processing The Transport Layer: Ideal Capabilities

Performance with Persistence Decoupling of Multiple Producers from Multiple Consumers

Streaming Data for a Microservices Architecture

Data Stream as the Centralized Source of Data Fraud Detection Use Case: Better Design with Stream-First Architecture Flexibility for Developers

Beyond Real-Time Applications Geo-Distributed Replication of Streams

3. What Flink Does

Different Types of Correctness

Natural Fit for Sessions Event Time Accuracy Under Failures: Keeping Track of State Answers When They Matter Ease of Development and Operations

Hierarchical Use Cases: Adopting Flink in Stages

4. Handling Time

Counting with Batch and Lambda Architectures Counting with Streaming Architecture

Batching in Stream Processing Systems

Notions of Time Windows

Time Windows Count Windows Session Windows Triggers Implementation of Windows

Time Travel Watermarks

How Watermarks Are Generated

A Real-World Example: Kappa Architecture at Ericsson

5. Stateful Computation

Notions of Consistency Flink Checkpoints: Guaranteeing Exactly Once Savepoints: Versioning State End-to-End Consistency and the Stream Processor as a Database Flink Performance: the Yahoo! Streaming Benchmark

Conclusion

6. Batch Is a Special Case of Streaming

Batch Processing Technology Case Study: Flink as a Batch Processor

A. Additional Resources

Going Further with Apache Flink

More on Time and Windows More on Flink’s State and Checkpointing Handling Batch Processing with Flink Flink Use Cases and User Stories Stream-First Architecture Message Transport: Apache Kafka Message Transport: MapR Streams

Selected O’Reilly Publications by Ted Dunning and Ellen Friedman