• Shell Commands
  • Bash
  • SSH
• Line-Commands Tools
  • Git/GitHub
    • Config and Init
    • Branch and Checkout
    • Stage area and commit
    • Remote
    • Push and Pull
    • Merge and Rebase
    • Large Files
  • Pipenv
  • Docker
  • Kubernetes and Kind
  • Others
    • Repository structure
    • jupyter
    • Jekyll
    • Xelatex

Copy the file to the destination repository:

cp /path/to/source/repo/file /path/to/destination/repo/

Rename a file in a repository:

mv old-file-name <new-file-name >

Remove all files and subdirectories in the current directory:

rm -rf *

Find the location of a program:

which <program-name >

Return the type of a program:

type <program-name >

List all the files installed by a program. The dpkg is the package management program that installs, removes, and provides information about .deb packages. The grep command-line searches through text and prints lines that match a pattern.

dpkg -L <program-name > | grep bin/

netstat is a command-line utility that displays network connections for the Transmission Control Protocol (both incoming and outgoing), routing tables, and a number of network interface (network interface controller or software-defined network interface) and network protocol statistics. The t flag is used to display TCP connections, the u flag is used to display UDP connections, the l flag is used to display only listening sockets, and the n flag is used to display numerical addresses instead of trying to determine symbolic host, port or user names. grep <port-number> is used to filter the displayed connections by a specific port number.

netstat -tuln | grep <port-number >

lsof stands for "list open files". Display information about files opened by processes. The i flag is used to display only internet connections, the p flag is used to display the PID and name of the program to which each socket belongs, and the n flag is used to display numerical addresses instead of trying to determine symbolic host, port or user names.

lsof -i -P -n | grep <port-number >

sudo lsof -i :9696

A cross-platform command-line utility that creates projects from cookiecutters (project templates):

cluster access (Password: marcos0102)

ssh -Y [email protected] -p  20022

CHange password:

passwd 

A "node" refers to a single computer or machine within the cluster. Each node can be a separate physical server, or it can be a virtual machine. To change of node in a cluster use:

ssh cn025

List all the nodes in the cluster:

sinfo -l

Displays information about the CPU architecture:

lscpu

Shows the amount of free and used memory in the system .

free -h

Information about the disk space usage on the cluster:

df -h

Gives detailed information about the GPU NVIDIA models, usage, memory:

nvidia-smi

Displays basic information about the system's kernel, operating system, and hardware platform:

uname -a

Commands show the current usage of CPU, memory, and other resources in real-time

top

or more user-friendly interface:

htop

List all the git configurations:

git config --list

git config --global --list

git config --local --list

Set the default branch name to main as in GitHub:

git config --global init.defaultBranch main

Initialize a git repository:

git init

Establish a new remote repository that our local repository can interact with:

git remote add origin <SSH_URL or HTTPS >

Difference between the working directory and the staging area:

git diff

List of news files and modified files:

git status

git branch is used for creating, listing, and deleting branches, while git checkout is used for switching between branches and also for creating a new branch if used with the -b flag.

List all the branches in the repository with -a flag:

git branch -a

or delete a branch with -d (-D to force deletion) flag:

git branch -d <branch-name>

Create and immediately switch to a new branch:

git branch -b <new-branch-name >

or

git checkout <new-branch-name >

Switch to an existing branch:

git checkout <branch-name >

Rename a branch:

git branch -m <old-branch-name> <new-branch-name>

Add a file to the staging area:

git add <file-name >

Remove a file from the staging area before commit:

git reset <file-name >

Commit changes to head:

git commit -m "Commit message"

Stop tracking a file that was previously committed to the repository. It's often used for files that should no longer be part of the repository (e.g., accidentally committed files, files that should be ignored).

git rm --cached <file-name>

Or remove all files from the staging area.The -r flag is for recursive removal, and . indicates the current directory.

git rm --cached -r .

Change the commit message or add/untrack files to last commit (only if not pushed and after staging/unstaging the files with git add / git rm):

git commit --amend -m "New commit message"

Show the commit history for the currently active branch:

git log

Push the branch to remote repository:

git push origin <branch-name >

Pull changes from the remote repository to the local repository:

git pull origin <branch-name >

Fetch the changes from the remote repository to the local repository:

git fetch origin <branch-name >

Merge the specified branch into the current branch:

git merge <branch-name >

Rebase the current HEAD onto the specified branch:

git rebase <branch-name >

Create a new commit that undoes all of the changes made in , then apply it to the current branch:

git revert <commit-hash >

Replace the current working directory and staging area with the state of the tree at the given commit:

git reset --hard [commit-hash]

If we only want to reset the staging area and not affect the working directory. This will unstage any changes since the specified commit, but leave the files in the working directory unchanged.

git reset [commit-hash]

Switch a repository's remote URL to use SSH in GitHub:

git remote set-url origin <SSH_URL >

List all the remote repositories:

git remote -v

initializes Git LFS in repository:

git lfs install

Track a file with Git LFS. It adds entries to the .gitattributes.

git lfs track <file-name >

Untrack a file with Git LFS:

git lfs untrack <file-name >

List all the files tracked by Git LFS:

git lfs ls-files

Install Pipenv to manage project dependencies.

pip install pipenv

Install a specific package and add it to Pipfile.

pipenv install <package-name >

Uninstall a specific package and remove it from Pipfile.

pipenv uninstall <package-name >

Activate the virtual environment associated with your project.

pipenv bash

Remove the virtual environment for the project.

pipenv --rm

Update all packages to their latest versions as specified in Pipfile.

pipenv update

Update a specific package to its latest version as specified in Pipfile.

pipenv lock

Exit the virtual environment:

exit

Download docker image:

sudo docker pull <docker-image >

List of docker images:

sudo docker images

Run docker image. The flags -it, it allows to interact with a command line interface within the Docker container. the --rm flag automatically removes the container when it exits and . specifies the build context to the current directory.

sudo docker run -it --rm -p 9696:9696 <docker-image >:tag .

List all containers (running and stopped). ps is used to list running process on unix and -a stands for 'all' process.

sudo docker ps -a

Inspect a docker object:

sudo docker inspect <image_or_container_id >

Remove a docker container forcefully:

sudo docker rm -f <container_id >

Remove a docker image forcefully:

sudo docker rmi -f <image_id >

Remove all stopped containers, not tagged images, and unused networks and volumes.

docker system prune -a

Create a cluster with kind:

kind create cluster

Check with kubectl that it was successfully created:

kubectl cluster-info

Load a docker image into the cluster:

kind load docker-image

Apply configuration to our cluster from a YAML file. The -f flag specifies the filename.

kubectl apply -f <filename >.yaml

Delete the Kubernetes resources defined in a given YAML file from our cluster:

sudo kubectl delete -f <filename >.yaml

List all deployments:

kubectl get deployments

Delete a deployment:

kubectl delete deployment <deployment-name >

List all pods:

kubectl get pods

List all services:

kubectl get services

Delete a service:

kubectl delete service <service-name >

Create the HPA (Horizontal Pod Autoscaler) for the deployment:

kubectl autoscale deployment <label-name-pod > --name <hpa-name > --cpu-percent=20 --min=1 --max=3

List all HPA:

kubectl get hpa

Show the details of a HPA:

kubectl describe hpa <hpa-name >

Delete HPA:

kubectl delete hpa <hpa-name >

Show the logs of the Metrics Server. Used to check the operational logs of the Metrics Server to diagnose issues, monitor its activities, or understand its interactions with other components in the cluster.

kubectl logs -n kube-system -l k8s-app=metrics-server

Stop the Kubernetes cluster managed by Kind, including all its control plane and worker nodes.

kind delete cluster --name your-cluster-name

cookiecutter https://github.com/drivendata/cookiecutter-data-science

Convert notebook to markdown:

jupyter nbconvert --to markdown <notebook-name>.ipynb

Create a new jekyll site and serve it locally:

jekyll new <site-name>

Run jekyll server locally:

bundle exec jekyll serve

Convert jupyter notebook to latex and then to pdf:

jupyter nbconvert --to latex <notebook-name >.ipynb

Convert latex to pdf:

xelatex <notebook-name>.tex