Title Page Copyright and Credits Learning Python Networking Second Edition About Packt Why subscribe? Packt.com Contributors About the authors About the reviewers Packt is searching for authors like you Preface Who this book is for What this book covers To get the most out of this book Download the example code files Download the color images Conventions used Get in touch Reviews Section 1: Introduction to Network and HTTP Programming Network Programming with Python Technical requirements An introduction to TCP/IP networking Introduction to TCP/IP The protocol stack, layer by layer UDP TCP Protocol concepts and the problems that protocols solve IP addresses and ports Network interfaces UDP versus TCP DHCP DNS Addressing NAT IPv4 IPv6 Python network programming through libraries An introduction to the PyPI Python repository Alternatives to pip for installing packages Conda Virtualenv Pipenv An introduction to libraries for network programming with Python Introduction to sockets Socket module in Python Client socket methods Server socket methods Working with RFC Extracting RFC information Downloading an RFC with urllib Downloading an RFC with requests Downloading an RFC with the socket module Interacting with Wireshark with pyshark Introduction to Wireshark Wireshark installation Capturing packets with Wireshark Network traffic in Wireshark Color coding in Wireshark Working with filters in Wireshark Filtering by protocol name HTTP objects filter Capture filters Display filters Analyzing networking traffic using the pyshark library FileCapture and LiveCapture in pyshark Summary Questions Further reading Programming for the Web with HTTP Technical requirements Consuming web services in Python with urllib Status codes Handling exceptions HTTP headers User agent Customizing requests with urllib Getting headers with a proxy Content types Extracting links from a URL with urllib Getting images from a URL with urllib Working with URLs Consuming web services in Python with requests Introduction to requests Checking HTTP headers Proxy requests Get whois information Working with JSON Handling forms with urllib and requests with Python 3.7 Handling forms with urllib Handling forms with requests Handling cookies with urllib and requests with Python What are cookies? Handling cookies with urllib Cookie handling with requests Handling HTTP Basic and Digest Authentication with requests Introduction to authentication mechanisms HTTP Basic authentication HTTP Digest authentication Summary Questions Further reading Section 2: Interacting with APIs, Web Scraping, and Server Scripting Application Programming Interface in Action Technical requirements Introduction to REST APIs Advantages of using REST APIs Introduction to JSON and the JSON module Encoding and decoding with the JSON package Using dict with JSON Interacting with a JSON hybrid-REST API (Twitter) The Twitter API Registering your application for the Twitter API Authenticating requests with OAuth Collecting information from Twitter A Twitter client Retrieving tweets from a timeline Searching tweets Consuming the Twitter REST API with Python Connecting with the Twitter API Accessing Twitter API resources Streaming APIs with Tweepy Introduction to XML Getting started with XML The XML APIs Processing XML with ElementTree Pretty printing Reading an XML file Working with XML and a full REST API (Amazon S3 bucket) with the Boto module The Amazon S3 API Registering with AWS Authentication with AWS S3 buckets and objects Creating a bucket with the S3 API Uploading and downloading file Listing buckets Parsing XML and handling errors Connecting to S3 with the Python Boto package Summary Questions Further reading Web Scraping with BeautifulSoup and Scrapy Technical requirements Introduction to web scraping Web content extraction What is web scraping? HTML parsers Parsing HTML with lxml Searching with XPath Extracting information from web pages and parsing HTML with BeautifulSoup BeautifulSoup introduction Access to elements through DOM Extracting labels using regex Handling URL exceptions and not found tags Introduction to Scrapy components and architecture What is Scrapy? Scrapy architecture XPath expressions Scrapy as a framework for performing web crawling processes and data analysis Installation of Scrapy Creating a project with Scrapy Scrapy item class Spiders Creating our spider Pipelines items and export formats Scrapy settings Executing Scrapy Scrapy execution tips and tricks EuroPython project Executing EuroPython spider Working with Scrapy in the cloud Scrapinghub Portia Start pages and link crawling Summary Questions Further reading Engaging with Email Technical requirements Introduction to email protocols Sending emails with SMTP through the smtplib library SMTP protocol Working with smtplib Sending a basic message Sending messages in HTML format Sending emails to multiple recipients Sending an email with attachments Authentication with TLS Establishing a connection with a Gmail SMTP server Using an external SMTP service Creating and sending an email with an attachment Learning the POP3 protocol and retrieving emails with poplib Understanding the POP3 protocol Introduction to poplib Retrieving emails with SSL Establishing a connection with Gmail for reading emails  Gmail account configuration Unread messages Manipulating and retrieving emails on the server email using IMAP with imapclient and imaplib IMAP protocol Retrieving emails with imaplib Retrieving emails with imapclient Summary Questions Further reading Interacting with Remote Systems Technical requirements Understanding the SSH protocol SSH introduction Using SSH to encrypt sessions How the SSH protocol works SSH service features Configuring the SSH protocol to make it more secure SSH terminals and running commands with paramiko Installing paramiko Establishing an SSH connection with paramiko Running commands with paramiko Running an interactive shell with paramiko SFTP with paramiko Paramiko alternatives Fabric Understanding the FTP protocol for transferring files The File Transfer Protocol Introduction to ftplib Other ftplib functions Inspecting FTP packets with Wireshark Reading and interacting with SNMP servers The SNMP  MIB – a broad base of information Introduction to pysnmp Polling information from the SNMP agent Reading and interacting with LDAP servers The LDAP protocol LDAP terminology Introduction to python-ldap The LDAP FreeIPA server Working with LDAP3 Accessing the LDAP server Finding entries in LDAP Summary Questions Further reading Section 3: IP Address Manipulation and Network Automation Working with IP and DNS Technical requirements Principles of the IP protocol Resolving the IP address with the socket package Validating the IP address with the socket package Retrieving the network configuration of a local machine Gathering information with the netifaces package Using Python to manipulate IP addresses and perform CIDR calculations The Python ipaddress module Manipulating IP addresses IP network objects Subnetting in Python Network interface objects IP address objects Planning IP addresses for your local area network The dnspython module as a tool for extracting information from DNS servers Working with dnspython Determining the destination of an MX record and its preference Manipulating domain names Converting IPv4 and IPv6 addresses into their DNS reverse map names Inspecting the DNS client and server communication GeoIP lookups with pygeoip and python-geoip Introduction to geolocation Introduction to pygeoip Introduction to python-geoip The MaxMind database in Python Summary Questions Further reading Implementing IPv6 and Address Manipulation Technical requirements Learning and understanding the IPv6 protocol The IPv6 protocol IPv6 addresses Representation of IPv6 addresses Reserved IPv6 addresses First steps with IPv6 – link-local Create an echo client and server with IPv6 Working with sockets The socket server The socket client Executing client and server Understanding netifaces module for checking IPv6 support on your network Introduction to netifaces Other packages for getting interfaces Using the netaddr module as a network-address manipulation library for Python Operating with IPv6 Understand ipaddress module as IPv4 and IPv6 manipulation library The Python ipaddress module IP network objects Subnetting in Python with IPv6 Network interface objects The IP address objects Planning IP addresses for your local area network Summary Questions Further reading Performing Network Automation with Python and Ansible Technical requirements Basics of Ansible Ansible introduction Installing Ansible Configuring Ansible Using Ansible Ansible's components and architecture Ansible's architecture Ansible's inventory file Automating network Python tasks with Ansible Ansible tasks Ad-hoc commands Using playbooks Writing Ansible modules with Python Introduction to Ansible modules Implementing Ansible modules with Python Summary Questions Further reading Section 4: Sockets and Server Programming Programming with Sockets Technical requirements Basics of sockets Sockets introduction Socket types Getting information about ports, protocols, and domains Creating a TCP client Banner grabbing with the socket module Port scanning with sockets Inspecting the client and server communication Working with UDP and TCP sockets in Python 3.7 Introduction to the TCP and UDP protocols Starting network programming with Python TCP sockets Starting a client Capturing packets in a loopback interface Inspecting the client and server interaction Code limitations Creating a simple UDP client and UDP server Implementing the UDP server Implementing the UDP client Working with IPv6 sockets in Python 3.7 Implementing the IPv6 server Implementing the IPv6 client Executing client and server Non-blocking and asynchronous socket I/O Introducing non-blocking I/O The client-server model with multiple connections HTTPS and securing sockets with TLS Implementing the SSL client Inspecting standard SSL client and server communication Summary Questions Further reading Designing Servers and Asynchronous Programming Technical requirements Building a multiprocessing-based TCP server Introducing the concurrent.futures module Application for checking websites The multiprocessing approach Building asynchronous applications with asyncio and aiohttp Introducing asyncio Using asyncio Introducing event loops Futures Task manipulation with asyncio Downloading files with asyncio Introducing aiohttp Downloading files with aiohttp Other event loop solutions Building asynchronous network applications with Tornado Introducing Tornado Implementing the Tornado web server Implementing an asynchronous client with AsyncHTTPClient Asynchronous generators Utilities in Tornado for asynchronous network operations Building asynchronous network applications with Twisted Introduction to Twisted Protocols Building a basic Twisted server Factory Reactor Building a socket client Executing the client and server Building a Twisted client Building a Twisted web server Building asynchronous network applications with Celery Celery architecture Installing Celery Installing Redis Introduction to Redis Distributing Python with Celery and Redis Summary Questions Further reading Designing Applications on the Web Technical requirements Writing a web application with WSGI Introducing WSGI Creating a WSGI application Existing web application frameworks (Django, Flask, and Plone) Web frameworks The MVC pattern and dynamic web programming with Python The MVC pattern Dynamic web pages Processing dynamic pages Accessing a database Django introduction Creating a Django application Creating RESTful web applications and working with Flask and HTTP requests Introducing Flask Routing in Flask Jinja2 templating POST parameters with Flask Other templating engines Flask extensions Working with a database in Flask with SQLAlchemy Introducing SQLAlchemy Creating a session and ORM queries Using Flask with SQLAlchemy Summary Questions Further reading Assessment Chapter 1, Network Programming with Python Chapter 2, Programming for the Web with HTTP Chapter 3, Application Programming Interface in Action Chapter 4, Web Scraping with BeautifulSoup and Scrapy Chapter 5, Engaging with Email Chapter 6, Interacting with Remote Systems Chapter 7, Working with IP and DNS Chapter 8, Implementing IPv6 and Address Manipulation Chapter 9, Performing Network Automation with Python and Ansible Chapter 10, Programming with Sockets Chapter 11, Designing Servers and Asynchronous Programming Chapter 12, Designing Applications on the Web Another Book You May Enjoy Leave a review - let other readers know what you think