I'll post scripts, etc., here that are helpful here.
# Purpose: A safe way to run rm -rf
# Author: Lex Sheehan
# License: MIT
Do you ever have that cringe in your tummy before running rm -rf? Worse yet, the face palm after running it?
I wrote a bash script named rmrf that allows me to safely delete directories. If anybody's interested, I'll make it accessible. Been using it for years w/o a mistake. BTW - I use rmrf while in the command line and typically use rm -rf in script files. Reference
This Ruby script will test the protocols for the paths for the domains specified in the .txt files.
Ever needed to verify that a list of domains have the paths and that they all respond to both http and https?
Assumes each .txt file's last line is empty
--------------
./paths.txt:
--------------
/about
/main
--------------
./protocols.txt:
--------------
https://
https://
--------------
./domains.txt:
--------------
google.com
yahoo.com
microsoft.com
Run the test script as follows:
$ bundle exec ruby paths_test.rb
NOTE: `gem install minitest-proveit` for even more hellish tests
Run options: --seed 60360
# Running:
Testing: https://google.com/about
Testing: https://google.com/mail
Testing: https://yahoo.com/about
Testing: https://yahoo.com/mail
Testing: https://microsoft.com/about
Testing: https://microsoft.com/mail
Testing: https://google.com/about
Testing: https://google.com/mail
Testing: https://yahoo.com/about
Testing: https://yahoo.com/mail
Testing: https://microsoft.com/about
Testing: https://microsoft.com/mail
.
Finished in 1.939989s, 0.5155 runs/s, 6.1856 assertions/s.
1 runs, 12 assertions, 0 failures, 0 errors, 0 skips
# Filename: info-github-labels
# Purpose: Info about how to label your git branches
Typical github branch labels:
* feature: Adding functionality? (may include tests, fixes, docs, etc.)
* fix : Fixing bugs. In production use 'hotfix'
* nfr : Functional Requirement, e.g., configuration
* test : Adding tests, e.g. unit tests, performance tests, etc.
* docs : Documentation, which may include design and/or install/config documention
* explore: An exploratory branch, which might get converted to a feature branch
Examples:
feature/add-get-customer-details-use-case
fix/use-correct-exchange-rates
hotfix/change-from-http-to-https
nfr/update-gemfile
test/add-perf-tests-for-fund-distribution
docs/add-wallet-use-cases
explore/try-acme-external-service
Use Tags to:
* Tag a commit with verison #. Ex: git tag -a v1.2.0 -m'v1.2.0 - Added Admin GUI' 9fceb02
* Tag steps in build pipeline. Ex: git tag -a step-03 -m'step 03 - After data migration'
When working by yourself, feel free to merge directly into master w/o using branches.
When working with a team you should create feature branches and open a pull request (PR) , s.t., others can review your code before merging into the projects' master branch.
Git branches should be small and short-lived.
The branch should not exist more than a few days, preferably only a few hours.
The longer you wait to create a PR the more likely extra work will be required to merge it with other(s) work.
If you are a team lead or project manager, do not allow a PR to sit idle; The longer the team waits to merge a PR, the more work you are creating for the person that created it. Do not allow team members with admin rights to circumvent your source code management process. While it might speed things up for your git admin/developer, the rest of your team will suffer.
The key to making feature branch based development work effectively is to not sit on your branch, i.e., avoid long-lived feature branches.
Only when you merge your feature branch to the master branch you are actually integrating your changes.
You can still use feature toggles, e.g. enabling different code paths for different users in the same deployed code.
In trunk-based development (TBD), developers always check into one branch, typically the master branch a/k/a the "trunk".
While TBD enables instant, continuous integration, however TBD:
- Leads to excessive use of feature toggles, which leads to code complexity
- Circumvents that PR code review process
- Requires an exhaustive suite of tests that are run prior to merging changes to master
TBD fosters more of a "get it out now" rather than a "quality control" mentality.
DevOps tends to like TDB better than Product Owners, Stakeholders, Architects and Developers.
Evaluate simple math expressions on the command line.
Parenthesis are not supported, but you can use $(expression)
with operators (+, -, *, /).
80 laps in a Jr. Olympic swimming pool is a mile.
2.5 miles is 200 laps. (2 miles + 1/2 mile.)
$ math 80*2
160
$ math 80*2+$(math 80/2)
200.00000000000000000000
Evaluate ruby code from the terminal.
If I plan to swim 1 lap, then 2 laps, then 3 laps.. (and back down)... 3 laps, 2 laps, 1 lap....
How many laps in my largest set if I want to swim at least 2.5 miles?
$ run-ruby-code "s=0;(1..14).each{|i| s+=i};puts s*2"
210
Find the most recently updated file in a directory.
Return most recently updated filename if no DIR passed, else full path.
Recurses the directory unless you pass the -maxdepth argument.
Return file name only for current directory if no DIR passed.
~/Downloads $ find-latest
goland-2019.3.3.tar.gz
Return file name only for current directory if no DIR passed.
~ $ find-latest ~/Downloads/
/home/lex/Downloads/goland-2019.3.3.tar.gz
This script allows you to bundle gems for your application based on groups in your Gemfile. (That's the "build" portion of this script.)
This script also allows you to test your rails environments based on RAILS_GROUPS. (That's the "load-gem-groups" portion).
This script can work with the default Bundler configuration for a generated Rails app in the config/application.rb file:
if defined?(Bundler)
# If you precompile assets before deploying to production, use this line
Bundler.require(*Rails.groups(:assets => %w(development test)))
# If you want your assets lazily compiled in production, use this line
# Bundler.require(:default, :assets, Rails.env)
end
Usage: build-and-load-gem-groups [-i group names] | [-e group names] | [-a] | [-d] | [-l] | [-r] | [-t]
Options:
-i or --include Groups to include in build and load
-e or --exclude Groups to exclude in build and load
-a or --all-groups Include all groups in build and load
-d or --display-groups Display all groups in Gemfile
-r or --clear-all Clear bundler and gemfile settings and remove vendor dir
-t or --test-load-time Display how long it takes to load
-l or --show-load-path Display Ruby load path
-c or --show-bundle-config Display bundle config file contents
-s or --save-config Currently only saves Gemile.lock file
-g or --last-gem-lock-file Display path of last saved Gemile.lock file
Examples:
build-and-load-gem-groups -i app development
build-and-load-gem-groups -i app test
build-and-load-gem-groups -i app production
build-and-load-gem-groups -i assets
build-and-load-gem-groups -i deploy
RAILS_GROUPS=development build-and-load-gem-groups -t
SKIP_BUILD=true build-and-load-gem-groups -i development
SKIP_LOAD=true build-and-load-gem-groups -i production
LOAD_ENV_PATH=./scripts/load-env.rb build-and-load-gem-groups -i development
Notes:
* A better, albeit longer name of this script: 'build_and_time_the_loading_of_gem_groups'
* - If group passed 'test' then 'rake rspec' will be run otherwise run load_environment.rb.
* In practice, you should set the RAILS_GROUPS before you run your Ruby code. [Optional]
* Set env var SKIP_BUILD=true to skip the build portion of this script. [Optional]
* Set env var SKIP_LOAD=true to skip the load portion of this script. [Optional]
* Set env var LOAD_ENV_PATH='path to where load_environment.rb path. [Optional]
* Run 'bundle info <GEM_NAME>' for info about installed gem. Ex: $ bundle info json
* Running $ build-and-load-gem-groups -i assets fails to load the rails environment as expected
because will likely be missing gems required to load a full rails environment.
* Running $ build-and-load-gem-groups -i development when RAILS_ENV=production may fail with:
'cannot load such file -- uglifier'
'cannot load such file -- scruffy'
'cannot load such file -- newrelic_rpm'
That is expected. In order to build a development package in production you must run:
$ build-and-load-gem-groups -i development app assets production
* Just because you can do something, does not mean you _should_ do it.
* If you see an error like the following, then install the test group passing '-i test'
'cannot load such file -- shoulda/matchers'
* If you see an error like the following after an --i operation, then install it's group
'cannot load such file -- ruby-hl7'
Assming its group is 'app', then run 'build-and-load-gem-groups -i test app'
When running bundle install
it takes *25.025s to load the Rails environment.
$ export RAILS_ENV=development && build-and-load-gem-groups -r && bundle install && build-and-load-gem-groups -t
The git source https://github.com/sr/shout-bot.git is not yet checked out. Please run `bundle install` before trying to start your application
Fetching https://github.com/RepoName/example_elastic
Fetching https://github.com/RepoName/notification-dispatch
Fetching https://github.com/rails/prototype_legacy_helper
. . .
Resolving dependencies...........................
RubyGems 1.8.23.2 is not threadsafe, so your gems will be installed one at a time. Upgrade to RubyGems 2.1.0 or higher to enable parallel gem installation.
Using rake 0.9.6
Using 12_hour_time 0.0.4
Using Ascii85 1.0.3
. . .
Using verhoeff 2.1.0
Using will_paginate 3.2.1
Using xml-simple 1.1.5
Bundle complete! 128 Gemfile dependencies, 209 gems now installed.
Gems in the groups Missing, Gemfile!, gui and deploy were not installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
Loading Rails environment...
boot.rb - Rails 3.2
/home/lex/.rbenv/versions/1.9.3-p551/lib/ruby/gems/1.9.1/gems/json_pure-1.4.6/lib/json/common.rb:2:in `<top (required)>': iconv will be deprecated in the future, use String#encode instead.
/home/lex/.rbenv/versions/1.9.3-p551/lib/ruby/gems/1.9.1/gems/12_hour_time-0.0.4/lib/12_hour_time/action_view_helpers.rb:3: warning: already initialized constant POSITION
DEPRECATION WARNING: The factory_girl gem is deprecated. Please upgrade to factory_bot. See https://github.com/thoughtbot/factory_bot/blob/v4.9.0/UPGRADE_FROM_FACTORY_GIRL.md for further instructions. (called from <top (required)> at /home/lex/Clients/Example/dev/example_22379_startup/config/application.rb:7)
Please configure your rails_lts_options using config.rails_lts_options inside Rails::Initializer.run. Defaulting to "rails_lts_options = { :default => :compatible }". See https://makandracards.com/railslts/16311-configuring-rails-lts for documentation.
DEPRECATION WARNING:
Sass 3.5 will no longer support Ruby 1.9.3.
Please upgrade to Ruby 2.0.0 or greater as soon as possible.
Rails.groups: [:default, "development"]
real 0m25.025s
user 0m23.368s
sys 0m1.077s
When running build-and-load-gem-groups -i
it takes 17.326s to load the Rails enviroment.
That's a 30% improvement in performance.
$ export RAILS_ENV=development && build-and-load-gem-groups -r && build-and-load-gem-groups -i
INCLUDES: development
EXCLUDES: app assets ci deploy test
>> Build groups_to_exclude=app assets ci deploy test
The git source https://github.com/sr/shout-bot.git is not yet checked out. Please run `bundle install` before trying to start your application
Fetching https://github.com/RepoName/example_elastic
Fetching https://github.com/RepoName/notification-dispatch
Fetching https://github.com/rails/prototype_legacy_helper
. . .
Using uuid 2.3.9
Using will_paginate 3.2.1
Using xml-simple 1.1.5
Bundle complete! 128 Gemfile dependencies, 160 gems now installed.
Gems in the groups app, assets, ci, deploy, test, Missing, Gemfile! and gui were not installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
SKIP_BUILD:
SKIP_LOAD:
--------------------------------------------------------
RAILS_GROUPS=development
--------------------------------------------------------
Loading Rails environment...
boot.rb - Rails 3.2
/home/lex/.rbenv/versions/1.9.3-p551/lib/ruby/gems/1.9.1/gems/json_pure-1.4.6/lib/json/common.rb:2:in `<top (required)>': iconv will be deprecated in the future, use String#encode instead.
/home/lex/.rbenv/versions/1.9.3-p551/lib/ruby/gems/1.9.1/gems/12_hour_time-0.0.4/lib/12_hour_time/action_view_helpers.rb:3: warning: already initialized constant POSITION
DEPRECATION WARNING: The factory_girl gem is deprecated. Please upgrade to factory_bot. See https://github.com/thoughtbot/factory_bot/blob/v4.9.0/UPGRADE_FROM_FACTORY_GIRL.md for further instructions. (called from <top (required)> at /home/lex/Clients/Example/dev/example_22379_startup/config/application.rb:7)
Please configure your rails_lts_options using config.rails_lts_options inside Rails::Initializer.run. Defaulting to "rails_lts_options = { :default => :compatible }". See https://makandracards.com/railslts/16311-configuring-rails-lts for documentation.
Rails.groups: [:default, "development"]
real 0m17.326s
user 0m16.309s
sys 0m0.947s
This script makes use of Rails groups to bundle and load Rails environments.
This will show you the groups you currently have defined in your Gemfile:
$ script/build-and-load-gem-groups -d
app
assets
ci
deploy
development
test
I wrote this script out of shear frustation.
It seems that I have a super power that allows me to find new ways to delete my work with git.
The irony is that git is supposed to help save and version your work, but every now and then after running a git command some of my work gets deleted and it's super frustrating.
Never again!
Before running any git command that can modify any file in my workspace, I run the following
git-changes -d
shows me what files I have modified and added on my feature branch.
The last line of output will the commit id that I use on the next backup command.
git-changes -b BACKUP_FROM_COMMIT_NO
saves my changes. You can change the default backup directory in 1st line of script:
root_dir=$HOME/REPOs/git-changes$HOME/REPOs/git-changes
Usage: git-changes [-b <BACKUP_FROM_COMMIT_NO>] [-c] [-d] [-r <UPSTREAM_BRANCH>] [-n]
Options:
-d or --display-changes Displays all commits since starting this branch (ends with previous commit hash)
-b or --backup-from-commit Backup new/changed files in current branch (pass previous commit hash from -d)
-c or --compare-with-backup Compare changed files in workspace with those in backup directory
-r or --rollback-changes Remove commits after BACKUP_FROM_COMMIT_NO (keeps your changes in your workspace)
-n or --copy-back-new-files Restore changed (new) files that you previously backed up
Examples:
git-changes -d
git-changes -b 2ed491b41b
git-changes -c
git-changes -r
git-changes -n
git-changes -b 2ed491b41b -r
git-changes -b 2ed491b41b -r -n
Notes:
* For details, run one of the following commands:
git log --pretty=one-line
git log --graph --pretty=format:'%Cred%h%Creset %an: %s - %Creset %C(yellow)%d%Creset %Cgreen(%cr)%Creset' --abbrev-commit --date=relative
git log --pretty=format:"%C(yellow)%h%Cred%d\ %Creset%s%Cblue\ [%cn]" --decorate --numstat
git log --graph --pretty=format:'%C(yellow)%h%Creset%C(cyan)%C(bold)%d%Creset %C(cyan)(%cr)%Creset %C(green)%ce%Creset %s'
This script was inspired from https://stackoverflow.com/questions/9393607/find-and-replace-filename-recursively-in-a-directory
lex@s76 ~ $ . tmp-make-dirs
lex@s76 ~/tmp/tmp.KWEdLcfN3Q $ tree --dirsfirst
.
├── 123_u
│ ├── 123_a.txt
│ ├── 123_b.txt
│ ├── exclude.me
│ ├── foo_123bar_123_d.txt
│ └── foo_123_c.txt
├── u
│ ├── 123_a.txt
│ ├── 123_b.txt
│ ├── exclude.me
│ ├── foo_123bar_123_d.txt
│ └── foo_123_c.txt
├── 123_a.txt
├── 123_b.txt
├── exclude.me
├── foo_123bar_123_d.txt
└── foo_123_c.txt
Now that files are in place, run the following command to recursively replace all file names that begin with '123_' with the same file name less the '123_'.
The '--' separates the command options from the positional arguments. The positional arguments are the arguments that you'd pass to a typical find command.
lex@s76 ~/tmp/tmp.kJzgPFEIQe $ find-rename-recursive --pattern '123_' --string '' -- . -type f -name "123_*"
lex@s76 ~/tmp/tmp.kJzgPFEIQe $ tree --dirsfirst
.
├── 123_u
│ ├── a.txt
│ ├── b.txt
│ ├── exclude.me
│ ├── foo_123bar_123_d.txt
│ └── foo_123_c.txt
├── u
│ ├── a.txt
│ ├── b.txt
│ ├── exclude.me
│ ├── foo_123bar_123_d.txt
│ └── foo_123_c.txt
├── a.txt
├── b.txt
├── exclude.me
├── foo_123bar_123_d.txt
└── foo_123_c.txt
2 directories, 15 files
Bash string subsitution was not working for -name "string" pattern. Latest commit fixes that.
lex@s76 ~/tmp/tmp.2zam8TNFqX $ . tmp-make-dirs
/tmp/tmp.vSrGgFxJX8 ~/tmp/tmp.2zam8TNFqX ~/tmp/tmp.6IOFgCwOyA ~/tmp/tmp.c1mr0dsaqG ~/tmp/tmp.GT29559NuB ~/REPOs/l3x/helpers
.
├── 123_u
│ ├── 123_a.txt
│ ├── 123_b.txt
│ ├── exclude.me
│ ├── foo_123bar_123_d.txt
│ └── foo_123_c.txt
├── u
│ ├── 123_a.txt
│ ├── 123_b.txt
│ ├── exclude.me
│ ├── foo_123bar_123_d.txt
│ └── foo_123_c.txt
├── 123_a.txt
├── 123_b.txt
├── exclude.me
├── foo_123bar_123_d.txt
└── foo_123_c.txt
2 directories, 15 files
Run popd to return to previous directory
lex@s76 ~/tmp/tmp.vSrGgFxJX8 $ mv 123_a.txt Max_a.txt
lex@s76 ~/tmp/tmp.vSrGgFxJX8 $ mv foo_123_c.txt foo_Max_c.txt
lex@s76 ~/tmp/tmp.vSrGgFxJX8 $ find-rename-recursive --pattern 'Max' --string 'RealBig' -- . -type f -name "*Max*"
lex@s76 ~/tmp/tmp.vSrGgFxJX8 $ tree --dirsfirst
.
├── 123_u
│ ├── 123_a.txt
│ ├── 123_b.txt
│ ├── exclude.me
│ ├── foo_123bar_123_d.txt
│ └── foo_123_c.txt
├── u
│ ├── 123_a.txt
│ ├── 123_b.txt
│ ├── exclude.me
│ ├── foo_123bar_123_d.txt
│ └── foo_123_c.txt
├── 123_b.txt
├── exclude.me
├── foo_123bar_123_d.txt
├── foo_RealBig_c.txt
└── RealBig_a.txt
2 directories, 15 files
All other solutions thus far either don't work for me and/or delete files with the sample I tried.
This solution deletes files!
lex@s76 ~/tmp/tmp.erUp1wWiSZ $ find . -name '123*.txt' -type f -exec bash -c 'mv "$1" "${1/\/123_//}"' -- {} \;
lex@s76 ~/tmp/tmp.erUp1wWiSZ $ tree --dirsfirst
.
├── 123_u
│ ├── exclude.me
│ ├── foo_123bar_123_d.txt
│ └── foo_123_c.txt
├── u
│ ├── a.txt
│ ├── b.txt
│ ├── exclude.me
│ ├── foo_123bar_123_d.txt
│ └── foo_123_c.txt
├── a.txt
├── b.txt
├── exclude.me
├── foo_123bar_123_d.txt
└── foo_123_c.txt
2 directories, 13 files
This solution just does not work:
lex@s76 ~/tmp/tmp.i9st1ZcHIn $ find -name "123*.txt" -exec rename 's/^123_//' {} ";"
lex@s76 ~/tmp/tmp.i9st1ZcHIn $ tree --dirsfirst
.
├── 123_u
│ ├── 123_a.txt
│ ├── 123_b.txt
│ ├── exclude.me
│ ├── foo_123bar_123_d.txt
│ └── foo_123_c.txt
├── u
│ ├── 123_a.txt
│ ├── 123_b.txt
│ ├── exclude.me
│ ├── foo_123bar_123_d.txt
│ └── foo_123_c.txt
├── 123_a.txt
├── 123_b.txt
├── exclude.me
├── foo_123bar_123_d.txt
└── foo_123_c.txt
2 directories, 15 files
This solution does not work recursively:
lex@s76 ~/tmp/tmp.Zg0G3AVCI9 $ rename 's/^123_//' *.txt
lex@s76 ~/tmp/tmp.Zg0G3AVCI9 $ tree --dirsfirst
.
├── 123_u
│ ├── 123_a.txt
│ ├── 123_b.txt
│ ├── exclude.me
│ ├── foo_123bar_123_d.txt
│ └── foo_123_c.txt
├── u
│ ├── 123_a.txt
│ ├── 123_b.txt
│ ├── exclude.me
│ ├── foo_123bar_123_d.txt
│ └── foo_123_c.txt
├── a.txt
├── b.txt
├── exclude.me
├── foo_123bar_123_d.txt
└── foo_123_c.txt
This solution works recursively when dirname does not include pattern, but DELETES FILES!
lex@s76 ~/tmp/tmp.xdK0oKW19B $ find . -name '123_*.txt'|awk '{print "mv "$0" "gensub(/\/123_(.*\.txt)$/,"/\\1","g");}'|sh
lex@s76 ~/tmp/tmp.xdK0oKW19B $ tree --dirsfirst
.
├── 123_u
│ ├── exclude.me
│ ├── foo_123bar_123_d.txt
│ └── foo_123_c.txt
├── u
│ ├── a.txt
│ ├── b.txt
│ ├── exclude.me
│ ├── foo_123bar_123_d.txt
│ └── foo_123_c.txt
├── a.txt
├── b.txt
├── exclude.me
├── foo_123bar_123_d.txt
└── foo_123_c.txt
2 directories, 13 files
lex@s76 ~/tmp/tmp.xdK0oKW19B $ # <= Is recursive when dirname does not include pattern, but DELETES FILES!
There are some scripts that don't have info in this README.md file.