Detaillierter Kursinhalt
Introduction
Workshop Application Use Cases
- Oz Metropolitan
- Architectural questions
- Team activity: Analyze Metroz
- Application Use Cases
Application Vertical Slice
- Definition
- Minimizing risk of an unsound architecture
- Selecting a vertical slice
- Team activity: Identify an initial vertical slice for Metroz
Application Processing
- Real time, near real time processing
- Batch processing
- Data access patterns
- Delivery and processing guarantees
- Machine Learning pipelines
- Team activity: identify delivery and processing patterns in Metroz, characterize response time requirements, identify Machine Learning pipelines
Application Data
- Three V’s of Big Data
- Data Lifecycle
- Data Formats
- Transforming Data
- Team activity: Metroz Data Requirements
Scalable Applications
- Scale up, scale out, scale to X
- Determining if an application will scale
- Poll: scalable airport terminal designs
- Hadoop and Spark Scalability
- Team activity: Scaling Metroz
Fault Tolerant Distributed Systems
- Principles
- Transparency
- Hardware vs. Software redundancy
- Tolerating disasters
- Stateless functional fault tolerance
- Stateful fault tolerance
- Replication and group consistency
- Fault tolerance in Spark and Map Reduce
- Application tolerance for failures
- Team activity: Identify Metroz component failures and requirements
Security and Privacy
- Principles
- Privacy
- Threats
- Technologies
- Team activity: identify threats and security mechanisms in Metroz
Deployment
- Cluster sizing and evolution
- On-premise vs. Cloud
- Edge computing
- Team activity: select deployment for Metroz
Technology Selection
- HDFS
- HBase
- Kudu
- Relational Database Management Systems
- Map Reduce
- Spark, including streaming, SparkSQL and SparkML
- Hive
- Impala
- Cloudera Search
- Data Sets and Formats
- Team activity: technologies relevant to Metroz
Software Architecture
- Architecture artifacts
- One platform or multiple, lambda architecture
- Team activity: produce high level architecture, selected technologies, revisit vertical slice
- Vertical Slice demonstration
Wrap Up