Log In
Or create an account ->
Imperial Library
Home
About
News
Upload
Forum
Help
Login/SignUp
Index
Title Page
Copyright and Credits
Big Data Architect's Handbook
Packt Upsell
Why subscribe?
PacktPub.com
Contributors
About the author
About the reviewers
Packt is searching for authors like you
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Download the color images
Conventions used
Get in touch
Reviews
Why Big Data?
What is big data?
Characteristics of big data
Volume
Velocity
Variety
Veracity
Variability
Value
Solution-based approach for data
Data – the most valuable asset
Traditional approaches to data storage
Clustered computing
High availability
Resource pooling
Easy scalability
Big data – how does it make a difference?
Big data solutions – cloud versus on-premises infrastructure
Cost
Security
Current capabilities
Scalability
Big data glossary
Big data
Batch processing
Cluster computing
Data warehouse
Data lake
Data mining
ETL
Hadoop
In-memory computing
Machine learning
MapReduce
NoSQL
Stream processing
Summary
Big Data Environment Setup
Oracle VM VirtualBox installation
Ubuntu installation
Hadoop prerequisite installation
Java installation
SSH installation and configuration
Hadoop system user
Apache Hadoop installation
Hadoop configuration
Path configuration for Hadoop commands
Hadoop server start and stop
Summary
Hadoop Ecosystem
Apache Hadoop
Hadoop Distributed File System
HDFS hands-on
Creating a directory in HDFS
Copying files from a local file system to HDFS
Copying files from HDFS to a local file system
Deleting files and folders in HDFS
Hadoop MapReduce
Job Tracker and Task Tracker
The execution flow of MapReduce
Mapper
Shuffle and Sort
Reducer
Example program
Preparing the data file for analysis
Program code
Driver program
Mapper program
Reducer program
Observations and results
YARN
Resource Manager
Node Manager
Container
Application Master
Apache Projects related to big data
Apache Zookeeper
Apache Kafka
Apache Flume
Apache Cassandra
Apache HBase
Apache Spark
Summary
NoSQL Database
What is NoSQL?
Benefits of NoSQL databases
NoSQL versus RDBMS
The CAP theorem
The ACID properties
Data models in NoSQL
Key-value data stores
Document store
Column stores
Graph stores
Apache Cassandra
Installation
Starting Cassandra
The Cassandra Query Language – CQL
The help command
Basic commands
Data manipulation
Creating, altering, and deleting a keyspace
Creating, altering, and deleting tables
Inserting, updating, and deleting data
The MongoDB database
Installing MongoDB
Starting MongoDB
Working on MongoDB
The help command
Basic commands
Data manipulation
Creating and deleting databases
Creating and deleting collections
The create, retrieve, update, delete operations
Neo4j database
Installing Neo4j
Starting Neo4j
The cypher query language
Help
Basic operations in Cypher
Creating nodes, relationships, and properties
Updating nodes, relationships, and properties
Deleting nodes, relationships, and properties
Reading nodes, relationships, and properties
Summary
Off-the-Shelf Commercial Tools
Microsoft Azure
Building a practical application
Microsoft Azure account
The Azure Event Hub
IoT simulation application
Setting up an Azure Stream Analytics job
Input
Query
Output
Dashboard in Power BI
Summary
Containerization
Virtualization
Hypervisors
Hardware-based hypervisors
Software-based hypervisors
What is containerization?
Benefits of containers
Docker
Docker workflow
Installation
Basic commands
Docker images
Building a Docker image
Running and verifying Docker images
Importing and exporting Docker images
Docker Swarm
Setting up Docker Swarm
Creating service containers
Replicating containers
Removing container services
Kubernetes
Key components
Pods
ReplicaSets
Deployments
PetSets
Installation
Deployment
Kubernetes Dashboard
Summary
Network Infrastructure
Network
Local area networks
Metropolitan area networks
Wide area networks
Network connectivity
Wired
Wireless
Network visualization
Gephi
Installation
Java installation
First run
Practical example
Summary
Cloud Infrastructure
Companies moving to cloud
Driving factors
Infrastructure
Locality of data
Requirements
Design considerations
Open source versus commercial
Commodity hardware versus purpose build
Cloud versus on-premises
Scale up and down
Application architecture
Cost decision
Summary
Security and Monitoring
Simple Network Management Protocol
Benefits of SNMP
Security
Agents and Traps
Netflow
Nagios
Key benefits
Security Onion
Deployment scenarios
The Standalone model
The Server-Sensor model
Hybrid model
Preconfigured tools
Wireshark
Key features
Summary
Frontend Architecture
React JS
Key concepts
Node.js
JSX
Unidirectional dataflow
Getting started with ReactJS
Single page application
React application project
React app directory structure
Components
Properties
Event handling
State
Redux
Architecture of Redux
Key concepts
Single store
Action
Reducers
Guestbook application
Installation
Create a store
Setting up Reducer
Setting up Dispatcher
Connect function
Setting up Subscribers
Final output
Summary
Backend Architecture
API
RESTful API
HTTP request methods
GET
POST
PUT
DELETE
Authentication
Basic authentication
JSON Web Token
Header
Payload
Signature
Practical
RESTful web service
Java client
Redis
Installation
Redis server
Redis client
Working with Redis
Redis data types and structures
String
HashMap
List
Set
Redis Publish/Subscribe
Common key operations
Summary
Machine Learning
Machine learning
Types of algorithms
Parametric algorithms
Non-parametric algorithms
Supervised learning
The classification model
Binary classification
Multi-class classification
The regression model
Linear regression
Polynomial regression
Unsupervised learning
Clustering, k-means
Neural networks
Feedforward neural network
Recurrent neural network
Symmetrically connected neural network
Deep neural networks
Decision tree classifiers
Summary
Artificial Intelligence
Artificial intelligence
Convolutional neural networks
Deep learning using TensorFlow
TensorFlow
Installation
TensorFlow program
Uninstalling TensorFlow
TensorBoard
Program
Launching TensorBoard
TensorBoard graph
Object detection using YOLO
Installation
Compiling YOLO library
Trained weights
Detecting objects in an image
Summary
Elasticsearch
Installing Elasticsearch
Starting the Elasticsearch server
Auto starting the Elasticsearch service
Stopping the Elasticsearch server
Uninstalling Elasticsearch
Kibana
Installation
Starting Kibana
Uninstalling Kibana
Security
Securing Elasticsearch
Securing Kibana
Understanding queries – CRUD commands
Creating
Reading
Updating
Deleting
Summary
Structured Data
Data analysis
Installing MySQL
Importing data
Analyzing the data model
HBase
Installation
Starting an HBase instance
Stopping a HBase instance
Preparing an HBase for migration
Sqoop
Installation
Verifying the installation
MySQL JDBC driver
Importing data
Verifying the imported data
Summary
Unstructured Data
Moving data into Hadoop
Downloading Flume
Environment configuration
Configuring agent and sink
Running Apache Flume
Transferring a log file
Converting images into text for analysis
Tesseract OCR
Installing Tesseract
Practical example
Complete code
Program execution
Summary
Data Visualization
Matplotlib
Installing Matplotlib
Line chart
Bar charts
Stack charts
Scatter charts
Pie charts
Geographic projections
D3.js
Installation
Practical example
Output
Summary
Financial Trading System
What is algorithmic trading?
Benefits of algorithmic trading
Big data in the financial market
Algorithmic trading strategies
Building an Expert Advisor
MetaTrader
Downloading and setting up MetaTrader
MetaQuotes language
Trading bot objective
Practical
Trading pattern – moving average
Decision time: buy or sell
Complete program
Backtesting in MetaTrader 4
Summary
Retail Recommendation System
Types of recommendation system
Collaborative filtering
Content-based filtering
Demographic-based system
Utility-based system
Knowledge-based system
Hybrid model
Commercial tools
Barilliance
Softcube
Strands
Monetate
Nosto
Book recommendation system
Dataset
Directory structure
Code
Reading the dataset
Verifying the dataset
Data analysis
Age group
Commutative rating
Algorithms
Top-rated books
Popular books
Demographic-based recommendation
Useful resources
Summary
Other Books You May Enjoy
Leave a review - let other readers know what you think
← Prev
Back
Next →
← Prev
Back
Next →