I log things that I learn or code snippets that I keep forgetting in this TIL collection.

This log also doubles as a lazy tech "blog". While I am the primary audience, I hope learning in public can offer utility to others.

• Python
• Statistics and Machine Learning
• SAS
• SQL
• C
• Linux
• Git
• Editors
• Resources
• Healthcare
• Other

• Pandas rename column inside method chain
• Pandas read_csv() from a file in ZIP URL
• Multithreading - running asynchronous steps that wait for other events to complete
• How to share Jupyter Notebooks: gists, nbviewer, binder
• Connecting and Posting Objects to AWS S3
• Catalog/inventory a file tree
• Get file owner on Windows
• Faker for Mock Data
• TF-IDF and cosine similarity implementation with scikit-learn
• Installing a package from Git repository, zip file, or PyPI
• Adding to PYTHONPATH environment variable so Python can find modules
• Convert markdown to HTML using markdown library
• Set Comparison Statistics
• Flatten nested list with generator
• Fill PDF Forms with PyPDF2
• Microsoft SQL Server (MSSQL) Connection String Example
• Decorators and functools.wraps
• Python Excel Recipes with openpyxl and xlwings
• Text-wrapping utility - textwrapper
• Why use classes?
• Merge mp4 movie files with moviepy.editor
• Get media file metadata with TinyTag
• API Usage Demo - FX Rates with Python Requests
• pandas - Pivot from columns to rows
• pandas - Display HTML representation of multiple dataframes
• Fast load to MSSQL / SQL Server from Python client
• Send Outlook emails with Win COM
• Install packages from jupyter with pip or conda
• Process large data sets by chunk in pandas
• pandas - Idiomatic Pandas: chaining, piping, inspecting and assigning intermediate dataframes
• pandas - Summarize Frequency Counts with Percentages by Column
• Create timezones for Python datetime objects
• OOP example
• OOP classes vs namedtuple
• pandas - Concatenate strings from rows with GroupBy
• Templating SQL from a pandas dataframe
• Dictionary un-packing and how to set default and override configurations
• Selecting data with df.query()
• Flatten a dictionary
• Iterative vs. Recursive Processes with Factorial
• Method chaining by returning self
• Flatten and remove pandas multi-index
• pivot non-numeric rows to x columns
• xlwings - add hyperlinks within a workbook
• pandas - miscellaneous
• Heron's Method for Square Roots
• Traverse file tree recursively
• Python Custom Logical Sort with sorted(iterable, key)
• Troubleshooting package installation SSL error on corporate firewall
• Neatly print a collection using using f-string padding and alignment or tabulate
• pandas - value counts with percentages snippet
• pandas - read multiple sheets from Excel
• Logging Format Snippet
• LRU Cache decorator
• Set up IPython kernel for VS Code Jupyter Notebook support with a virtual environment
• pandas - display dataframe without index
• pandas - debug dataframe inside method chain
• regex - split on delimiters and handle escape characters
• AWS and pandas - read a CSV object from a private S3 bucket
• Behavior of positional vs keyword arguments, defaults, and enforcing usage
• Design Patterns: Factory Method / Virtual Constructor
• Packaging non-Python files with a Python package (ex: sql, json, etc.)
• pytest - set up local Postgres DB for testing with Docker
• pytest - Software Testing with Pytest Using Mocks and Fixtures

TODO

• TODO: Use of environment variables to protect credentials
• TODO: xlwings - read & write with Excel; split & merge sheets and workbooks
• TODO: matplotlib - subplots
• TODO: pandas pivoting methods and "named aggregations"
• TODO: mixin inheritance pattern
• TODO: class decorators
• TODO: decorator demo; timer and logger
• TODO: closure demo
• TODO: generator and generator expressions to process large file
• TODO: making a class support Python container and iteration protocols
• TODO: producer-consumer with and without queue
• TODO: threading, thread pool, locks, and parallel loops
• TODO: async demo with timer/simulated SQL (long query)
• TODO: add steps for PyPi prep; modifying setup.py, sdist and bdist_wheel; twine upload to test, etc.
• TODO: using pandas dataframe index for faster lookups and filtering
• TODO: include data files in packages with pkgutil.get_data(package, resource)

• Simple Linear Regression with Python statsmodels vs R lm() function
• Linear Model Gotchas with Python vs R
• TODO: scikit-learn pipelines
• TODO: scikit-learn column tranformers; feature unions of categorical and numeric
• TODO: scikit-learn subclassing and custom transformers (e.g., fasttext text classifier; customer transformations)
• TODO: compare linear regression in statsmodels vs scikit-learn
• TODO: compare FastText vs scikit-learn vs SpaCy for text classification

• Configuring SASPy for Windows
• SAS Loops
• Expand date spans to year-month view
• Comparing terminology for similar concepts in Python and SAS
• Disambiguating double ampersand and period in SAS macro variables
• RSUBMIT and asynchronous SAS tasks
• Export SAS to Excel example

• psycopg2 connection and cursor commit behavior for PostgreSQL
• Programmatically generating SQL with Jinja Templates
• Connect to MS SQL Server with pyodbc
• PostgeSQL connection from Python
• PostgreSQL fast bulk load via copy from
• PostgreSQL update and delete joins
• PostgreSQL - list roles and permissions
• TODO: pivot, unpivot

• Tic Tac Toe CLI

• TODO: systemd - deaemon process control config example
• supervisorctl - daemon process control config example
• nginx sample web server config
• Check processes
• Check listening ports
• nohup - No Signal Hangup to run processes in background
• Inspect and persist memory usage statistics with vmstat
• Use grep to search for source files with lines containing words that match a pattern

• Show list of files with diffs since latest commit
• Git snippets
• .gitignore snippet

• VS Code - configuration for Anaconda prompt
• How to Remote SSH with sudo privilege in VS Code
• Step into code of third-party packages with debugger
• VS Code Extensions

• So You Want to Learn Python... (Python resource list)

• Data Anonymization Fundamentals for Data Science

• Data cleaning
  • PrettyPandas
  • tabulate
  • scrubadub
  • ftfy(fixes text for you)
• Code quality and testing
  • pytest
  • pytest plugins
    • pytest-xdist - distributed testing
    • pytest-cov - measure test coverage
    • pytest-randomly - randomly order tests and control random.seed to reduce flaky tests
    • pytest-freezegun - freeze time by mocking the datetime module; pytest wrapper around freezegun
    • pytest-sugar - prettier display of test output and progress bar
  • moto - mock out tests based on AWS infrastructure
  • hypothesis - property-based testing
  • black - formatting
  • ruff - linting
  • unimport - remove un-used imports

• CMS Risk Adjustment Files
• Implementing HEDIS Continuous Enrollment Requirements

• Make diagrams from markdown-like text with Mermaid
• Cracking the Caesar Cipher with relative frequency analysis