SAAD
Email: [email protected]
Website: https://engr-saad.com
LinkedIn: https://www.linkedin.com/in/raselhasandurjoy
Facebook: https://www.facebook.com/raselhasandurjoy
Company WEB: https://triangeltech.com/
Phone: +8801904654712

Recently, many friends who want to learn Python have joined our communication group one after another. At present, the number of our communication group has exceeded 10,000. Our goal is to create a high-quality Python communication community. On the one hand, it clears the obstacles in the entry process for beginners who want to learn Python; on the other hand, it provides a way for new developers to ask questions and help them quickly grow into excellent ones. Professionals; In addition, experienced developers can use this platform to share or provide their work experience for free, so that everyone can get a comprehensive promotion of professional skills and comprehensive quality. The previous public classes and offline technical exchange activities have been abandoned for some time because of work. However, the small partners are still active in the communication group and support us as always, thank you here. Recently, the content of the first 15 days and the last 10 days has been continuously updated. The first 15 days are written for beginners. I hope that the difficulty of getting started will be further reduced, and the example program will be simpler and clearer. The last 10 days are related to the actual combat of Python projects and interviews. I hope that the content is more detailed and complete, especially the interview question on the 100th day. It is not easy to create. Thank you for your support. The money will not be used for coffee purchases but will be donated to the people in need through the Tencent public welfare platform. ( Click for donations).

Simply put, Python is an "elegant", "clear", and "simple" programming language.

Low learning curve, even non-professionals can get started Open source system with a strong ecosystem Interpreted language, perfect platform portability Supports object-oriented and functional programming Ability to extend functionality by calling C / C ++ code Code is highly standardized and readable Python is useful in several popular areas.

Cloud Infrastructure-Python / Java / Go DevOps-Python / Shell / Ruby / Go Web crawler-Python / PHP / C ++ Data Analysis Mining-Python / R / Scala / Matlab Machine Learning-Python / R / Java / Lisp As a Python developer, the main employment areas include:

Python server background development / game server development / data interface development engineer Python Automation Operation and Maintenance Engineer Python data analysis / data visualization / big data engineer Python crawler engineer Python chat robot development / image recognition and vision algorithm / deep learning engineer

A few suggestions for beginners:

Make English as your working language. Practice makes perfect. All experience comes from mistakes. Don't be one of the leeches. Either stand out or kicked out. Day01 ~ 15- Python language basics

Introduction to Python-History of Python / Pros and Cons of Python / Application Areas of Python Build a programming environment-Windows environment / Linux environment / MacOS environment Run Python program from terminal-Hello, world / print function / run program Using IDLE-Interactive Environment (REPL) / Writing Multiple Lines of Code / Running a Program / Exiting IDLE Comments-The role of comments / single-line comments / multi-line comments

Programs and Bases-Instructions and Programs / Von Neumann Machines / Binary and Decimal / Octal and Hex Variables and types-variable naming / use of variables / input functions / checking variable types / type conversion Numbers and strings-integers / floating points / complex numbers / strings / string basic operations / character encoding Operators-Mathematical Operators / Assignment Operators / Comparison Operators / Logical Operators / Identity Operators / Operator Priorities Application Case-Convert Fahrenheit to Celsius / Enter the radius of the circle to calculate the perimeter and area / Enter the year to determine if it is a leap year

Application Scenarios for Branching Structure-Conditions / Indentation / Code Blocks / Flowchart if statement-simple if / if-else structure / if-elif-else structure / nested if Application case-User authentication / Swap the imperial and metric units / Roll the dice to decide what to do / Percentage to grade system / Piecewise function evaluation

Application Scenarios for Loop Structures-Conditions / Indentation / Code Blocks / Flowcharts while loop-basic structure / break statement / continue statement for loop-basic structure / range type / branch structure in loop / nested loop / early end of program Application Cases-1 ~ 100 Summation / Judging Prime Numbers / Guessing Numbers Game

Classic case: Number of daffodils / hundreds of chickens / Craps gambling game Practice questions: Fibonacci sequence / perfect number / prime number

The role of functions-bad smell of code / function modules with functions Define function-def statement / function name / parameter list / return statement / call custom function Calling Functions-Python Built-in Functions / Importing Modules and Functions Function arguments-default arguments / variable arguments / keyword arguments / named keyword arguments Function return value-no return value / single value returned / multiple values ​​returned Scope issues-local scope / nested scope / global scope / built-in scope / scope-related keywords Manage functions with modules-module concepts / functions with custom modules / naming conflicts (same module and different modules)

Use of Strings-Calculating Length / Subscript Operation / Slicing / Common Methods Basic usage of lists-define lists List common operations-join / copy (copy elements and copy arrays) / length / sort / reverse / find Generate List-create a list of numbers using range / generate expressions / generator Use of tuples-define tuples / use values ​​in tuples / modify tuple variables / tuples and list conversion Collection basic usage-the difference between collection and list / create collection / add element / delete element / empty Common Set Operations-Intersection / Union / Difference / Symmetric Difference / Subset / Superset Basic dictionary usage-Features of dictionary / Create dictionary / Add element / Remove element / Value / Clear Dictionary common operations-keys () method / values ​​() method / items () method / setdefault () method Basic Exercises-Marquee Effects / Finding the Largest Element in a List / Statistical Average of Test Results / Fibonacci Sequence Comprehensive case-Two-color ball selection / Tic-tac-toe

Classes and Objects-What Are Classes / What Are Objects / Other Object Oriented Concepts Defining Classes-Basic Structure / Properties and Methods / Constructors / Destructors / str Methods Working with objects-creating objects / messaging objects Four Pillars of Object Orientation-Abstraction / Encapsulation / Inheritance / Polymorphism Basic Exercises-Define Student Class / Define Clock Class / Define Graphics Class / Define Automotive Class

Properties-class properties / instance properties / property accessors / property modifiers / property deleters / use slots Methods in the class-instance methods / class methods / static methods Operator overloading-add / sub / or / __ getitem__ / setitem / len / repr / gt / lt / le / ge / eq____________ _______ Relationship between classes (objects)-association / inheritance / dependency Inheritance and Polymorphism-What is Inheritance / Inheritance Syntax / Calling Parent Method / Method Override / Type Decision / Multiple Inheritance / Diamond Inheritance (Diamond Inheritance) and C3 Algorithm Comprehensive case-salary settlement system / automatic book discount system / custom scores

Develop GUI programs using tkinter Use pygame library to develop game applications "Big Ball Eat Small Ball" game

Read file-read entire file / read line by line / file path Write file-overwrite / append write / text file / binary file Exception Handling-Importance of Exception Mechanisms / try-except code blocks / else code blocks / finally code blocks / built-in exception types / exception stacks / raise statements Data Persistence-Overview of CSV files / Application of csv module / JSON data format / Application of json module

Advanced string operations-escape characters / raw strings / multi-line strings / in and not in operators / methods starting with is / join and split methods / strip related methods / pyperclip module / immutable strings and variable characters Use of String / StringIO Getting started with regular expressions-What regular expressions do / metacharacters / escapes / quantifiers / grouping / zero-width assertions / greedy matching and lazy matching laziness / regular expression operations (match, search, replace, capture) with re Use regular expressions-re modules / compile functions / group and groups methods / match methods / search methods / findall and finder methods / sub and subn methods / split methods Use Case-Validate Input Strings Using Regular Expressions

The concept of processes and threads-what is a process / what is a thread / multi-threaded application scenario Use process-fork function / multiprocessing module / process pool / inter-process communication Using threads-thread module / threading module / Thread class / Lock class / Condition class / Thread pool

Computer Network Basics-History of Computer Networks / "TCP-IP" Model / IP Address / Port / Protocol / Other Related Concepts Web Application Mode-Client-Server Mode / Browser-Server Mode Access network resources based on HTTP protocol-Overview of network API / Access URL / requests module / Parse JSON format data Python Network Programming-Socket Concept / Socket Module / Socket Function / Create TCP Server / Create TCP Client / Create UDP Server / Create UDP Client / SocketServer Module Email-SMTP protocol / POP3 protocol / IMAP protocol / smtplib module / poplib module / imaplib module SMS Service-Call SMS Service Gateway

Process pictures with Pillow-picture reading and writing / picture synthesis / geometric transformation / color conversion / filter effects Reading and Writing Word Documents-Handling of Text Content / Paragraphs / Headers and Footers / Style Handling Read and write Excel files-xlrd module / xlwt module Generate PDF file-pypdf2 module / reportlab module

Common data structures Advanced Use of Functions-"First Class Citizens" / Higher Order Functions / Lambda Functions / Scopes and Closures / Decorators Advanced Object-Oriented Knowledge-"Three Pillars" / The Relationship Between Classes / Garbage Collection / Magic Properties and Methods / Hybrids / Metaclasses / Object-Oriented Design Principles / GoF Design Patterns Iterators and generators-related magic methods / Two ways to create generators / Concurrent and asynchronous programming-multi-threaded / multi-process / asynchronous IO / async and await

Host page content with HTML tags Rendering pages with CSS Handling interactive behavior with JavaScript Getting started with jQuery Getting started with Vue.js Use of Element Use of Bootstrap

Operating system history and Linux overview Linux basic commands Utilities in Linux Linux file system Vim editor application Environment variables and shell programming Software installation and service configuration Network access and management Other related content

Relational Database MySQL Overview of relational databases Installation and use of MySQL Use of SQL

DDL-Data Definition Language-create / drop / alt

DML-Data Manipulation Language-insert / delete / update / select

DCL-Data Control Language-grant / revoke

related information

Paradigm Theory-Guidelines for Designing Two-Dimensional Tables

Data integrity

Data consistency

Manipulating MySQL in Python

Getting started with NoSQL

NoSQL overview

Redis overview

Mongo overview

Web application working principle and HTTP protocol Overview of the Django framework Get started in 5 minutes Using view templates

Relational database configuration Use of management background CRUD operations on models using ORM Django model best practices Model definition reference

Load static resources Get data with Ajax request

Forms and form controls Cross-site request forgery and CSRF token Form and ModelForm form validation

Implementing user tracking The relationship between cookies and sessions Django framework support for sessions Read and write cookies in view functions

Modify the response header through HttpResponse Handling large files with StreamingHttpResponse Generate Excel report using xlwt Generate PDF report using reportlab Generate front-end charts with ECharts Configuration logs and Django-Debug-Toolbar

What is middleware Middleware built into the Django framework Custom middleware and its application scenarios

Returns data in JSON format Rendering a page with Vue.js

Website optimization first law

Implement page caching using decorators

Provide caching services for data interfaces

File upload form control and picture file preview How the server handles uploaded files Rich text editor overview Use of wangEditor

Introduction to common SMS gateway platforms Send a text message with a screw cap Django framework support for mail services

Website optimization second law Configure the message queue service Use celery to make tasks asynchronous in projects Use celery to implement timed tasks in projects

Unit tests in Python Django framework support for unit testing Using a version control system Configure and use uWSGI Dynamic and static separation and Nginx configuration Configure HTTPS

Concurrent programming I / O mode and event-driven

Tornado overview Get started with Tornado in 5 minutes Route resolution Request handler

Use of aiomysql and aioredis

WebSocket Introduction

WebSocket server-side programming

WebSocket client programming

Project: Web chat room

Day65- Project Combat

Front-end and back-end separation development and interface documentation Front-end rendering with Vue.js Use ECharts for reporting Using WebSocket for Push Services

The concept of web crawler and its application field Discussion on the legitimacy of web crawlers Related tools for developing web crawlers The composition of a crawler

Data Acquisition Standards and Tripartite Libraries Three ways of page parsing: regular expression parsing / XPath parsing / CSS selector parsing

How to store massive data Implementing data caching

Multi-threaded and multi-process Asynchronous I / O and coroutines Use of async and await keywords Application of three party library aiohttp

JavaScript reverse engineering Get dynamic content with Selenium

Auto submit form Cookie pool application Verification code processing

Overview of the Scrapy crawler framework Install and use Scrapy

Spider Application of middleware: download middleware / spider middleware Scrapy docks with Selenium to grab dynamic content Scrapy deployment to Docker

The principle of distributed crawlers Scrapy distributed implementation Distributed deployment with Scrapyd

Crawling Job Site Data Crawling Real Estate Industry Data Crawling Used Car Trading Platform Data

Software process model

Feasibility analysis (do or don't do research), output "feasibility analysis report". Requirement analysis (research what to do), output "Requirement Specification" and product interface prototype diagram. Summary design and detailed design, output conceptual model diagram (ER diagram), physical model diagram, class diagram, timing diagram, etc. Coding / testing. Go Live / Maintenance. The biggest shortcoming of the waterfall model is that it cannot embrace changes in demand. The product can only be seen after the entire process is completed, and the morale of the team is low.

Scrum-Product Owner, Scrum Master, Developer-Sprint

Product Backlog (user story, product prototype).

Planning meetings (evaluation and budget).

Daily development (stand-up meetings, tomato work, pair programming, test-first, code refactoring ...).

Fix bugs (problem description, recurring steps, testers, assignees).

release version.

Review meeting (Showcase, users need to participate).

Retrospective (to summarize the current iteration cycle).

Added: Manifesto for Agile Software Development

Individuals and interactions above processes and tools working software above detailed documentation Customer cooperation above contract negotiation. Response to change is higher than following plan

Preparations: business case and funding, contract, vision, initial product requirements, initial release plan, shareholding, team formation.

Agile teams usually have 8-10 people.

Estimation of workload: Quantify development tasks, including prototypes, logo design, UI design, front-end development, etc. Try to break down each task to the minimum task amount, the minimum task amount standard is that the working time cannot exceed two days, and then estimate the overall project time . Stick each task on the kanban, which is divided into three parts: to do (to be completed), in progress, and done.

Project team formation

Team composition and role

Note: Thank you, Ms. Fu Xiangying, for drawing this beautiful organization chart.

company_architecture

Programming specifications and code review (flake8, pylint)

Some "conventions" in Python (refer to "Python Conventions-How to Write Pythonic Code" )

Reasons affecting code readability:

Too few or no comments Code breaks language best practices Anti-pattern programming (spaghetti code, copy-paste programming, conceit programming, ...) Team Development Tools

Version control: Git, Mercury Defect Management: Gitlab , Redmine Agile closed-loop tools: Zen Tao , JIRA Continuous integration: Jenkins , Travis-CI Please refer to "Problems and Solutions for Team Project Development" .

Project selection and understanding business Topic selection range setting

CMS (user terminal): news aggregation site, Q & A / share community, film review / book review site, etc.

MIS (user side + management side): KMS, KPI assessment system, HRS, CRM system, supply chain system, warehouse management system, etc.

App background (management side + data interface): second-hand transactions, newspapers and magazines, niche e-commerce, news and information, travel, social, reading, etc.

Other types: own industry background and work experience, business is easy to understand and control.

Requirements understanding, module division and task allocation

Need understanding: brainstorming and competitive product analysis. Module division: draw a mind map (XMind), each module is a branch node, and each specific function is a leaf node (expressed by a verb). You need to ensure that each leaf node cannot reproduce a new node and determine each leaf Node importance, priority, and workload. Task assignment: The project leader assigns tasks to each team member according to the above indicators.

Module Features personnel status carry out Working hours Start of plan Actual start End of plan Actual end Note comment add comment King Sledgehammer Ongoing 50% 4 8/7/2018 8/7/2018 Delete comment King Sledgehammer wait 0% 2 8/7/2018 8/7/2018 View comments Bai Yuanfang Ongoing 20% 4 8/7/2018 8/7/2018 Need code review Comment vote Bai Yuanfang wait 0% 4 8/8/2018 8/8/2018 OOAD and database design

Class diagram of UML (Unified Modeling Language)

uml

Creating a table from a model (forward engineering)

python manage.py makemigrations app python manage.py migrate Use PowerDesigner to draw physical model diagrams.

python manage.py inspectdb > app / models.py

Introduction to Docker Install Docker Create containers using Docker (Nginx, MySQL, Redis, Gitlab, Jenkins) Building a Docker image (writing Dockerfile and related instructions) Container orchestration (Docker-compose) Cluster management

Public issues in project development Database configuration (multi-database, master-slave replication, database routing) Configuration of the cache (partition cache, key settings, timeout settings, master-slave replication, failure recovery (sentinel)) Configuration of logs Analysis and debugging (Django-Debug-ToolBar) Easy-to-use Python modules (date calculation, image processing, data encryption, tripartite API) REST API design RESTful architecture Understanding RESTful architecture RESTful API Design Guide RESTful API best practices Writing of API interface documents RAP2 YAPI django-REST-framework application Analysis of key and difficult points in the project Using caching to ease database pressure-Redis Decoupling and peak clipping using message queues-Celery + RabbitMQ

unit test Type of test Writing unit tests (unittest, pytest, nose2, tox, ddt, ...) Test coverage Project deployment Preparations before deployment Key settings (SECRET_KEY / DEBUG / ALLOWED_HOSTS / cache / database) HTTPS / CSRF_COOKIE_SECUR / SESSION_COOKIE_SECURE Log related configuration Linux common commands review Installation and configuration of common services in Linux Use of uWSGI / Gunicorn and Nginx Comparison of Gunicorn and uWSGI For simple applications that do not require a lot of customization, Gunicorn is a good choice. The learning curve of uWSGI is much steeper than Gunicorn. The default parameters of Gunicorn can already adapt to most applications. uWSGI supports heterogeneous deployment. Because Nginx natively supports uWSGI, Nginx and uWSGI are usually bundled together for deployment online, and uWSGI is a fully functional and highly customized WSGI middleware. In terms of performance, Gunicorn and uWSGI actually perform similarly. Deploy test and production environments using virtualization technology (Docker) Performance Testing Use of AB Use of SQLslap Use of sysbench automated test Automate testing with Shell and Python Automate testing with Selenium Selenium IDE Selenium WebDriver Selenium Remote Control Introduction to the testing tool Robot Framework

MySQL database tuning Web server performance optimization Nginx load balancing configuration Keepalived for high availability Code performance tuning Multithreading Asynchronous Static resource access optimization Cloud storage CDN