pszemraj / cautious_adamw.py

Last active November 27, 2024 03:53

Implements Cautious AdamW optimizer by subclassing AdamW https://github.com/kyleliang919/C-Optim

	import math
	import torch
	from torch.optim.adamw import AdamW


	class CautiousAdamW(AdamW):
	"""
	Implements Cautious AdamW optimizer by subclassing AdamW.

	All hyperparameters remain identical to AdamW.

pszemraj / chibihash64.py

Created November 18, 2024 16:38

Compute the 64-bit hash of the input bytes object using the chibihash64 algorithm.

	def chibihash64__load64le(p, offset=0):
	"""Load 8 bytes from the input bytes object as a little-endian 64-bit integer."""
	return int.from_bytes(p[offset : offset + 8], "little", signed=False)


	def chibihash64(key, seed=0):
	"""
	Compute the 64-bit hash of the input bytes object using the chibihash64 algorithm.

	Parameters:

pszemraj / complex_exponential_moving_average.py

Created November 11, 2024 03:00

working CEMA for cpu

	"""
	based on the megalodon implementation

	https://github.com/XuezheMax/megalodon/blob/53cbaa3a3b3e05ea439564bd67cb352012ba6b97/megalodon/modules/complex_exponential_moving_average.py
	"""

	import math
	import torch
	from torch import nn
	from typing import Optional, Tuple

pszemraj / pdf2markdown_openai.py

Created October 11, 2024 03:40

transcribe PDF files to markdown text with openai

	import base64
	import tempfile
	from pathlib import Path

	import fire
	import mdformat
	from joblib import Memory
	from openai import OpenAI
	from pdf2image import convert_from_path
	from tqdm.auto import tqdm

pszemraj / visualize_t5_weight_diffs.py

Created October 3, 2024 02:54

compare two t5 models of the same arch

	from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
	import torch
	import matplotlib.pyplot as plt
	import seaborn as sns

	# Define model names for base and fine-tuned versions
	base_model_name = "pszemraj/tFINE-900m-e16-d32-1024ctx"
	ft_model_name = "BEE-spoke-data/tFINE-900m-e16-d32-instruct_2e"

	# Load the models with appropriate dtype

pszemraj / join_files2md.py

Last active September 27, 2024 00:57

join a bunch of text-based files into a single markdown doc (for LLM prompt or otherwise)

	"""
	Merges all text files in a directory into a single markdown file.
	"""

	import logging
	import re
	from datetime import datetime
	from pathlib import Path

	import fire

pszemraj / calibration_datav3.txt

Created September 24, 2024 00:31 — forked from bartowski1182/calibration_datav3.txt

Calibration data provided by Dampf, combines his own efforts on top of Kalomaze's. Used for calibrating GGUF imatrix files

	In addition to a significant decrease in hepatic lipid accumulation in the IOE group, which inhibited energy intake by propionate enrichment, hepatic lipids were also significantly reduced in the mice in the IOP group, which was largely enriched with butyrate. Compared with the IOE group, IOP had a stronger regulatory effect on hepatic metabolism and triglyceride metabolism and higher levels of TCA cycle in the host. In addition, butyrate has the ability to promote browning of white adipose tissue (WAT) to brown adipose tissue (BAT).^[@ref39],[@ref40]^ WAT stores energy, whereas BAT uses energy for heating and consequently host energy expenditure increases.^[@ref41],[@ref42]^ However, adipose tissue weight does not change after WAT browning.^[@ref43]^ Therefore, the weight of adipose tissue of mice in the IOP group dominated by butyrate was greater than that of the mice in the IOE group dominated by propionate.

	In conclusion ([Figure [7](#fig7){ref-type="fig"}](#fig7){ref-type="fig"}C), the improvement of ob

pszemraj / whistleblowing_ai.md

Created September 13, 2024 02:46

a guide to being a whistleblower for employees of "Open"AI, Anthropic, and Google

whistleblowing_ai.md

Whistleblowing within AI companies, especially concerning sensitive information like model sizes and training methods, is a significant and serious action. It requires careful consideration of the legal, ethical, and personal implications involved. This guide aims to provide employees with comprehensive information on safely and anonymously whistleblowing or leaking confidential AI-related data while minimizing personal and professional risks.

	# !pip install -q sentence-splitter
	import os
	from sentence_splitter import split_text_into_sentences

	REFUSAL_TERMS = [
	"sorry",
	"i can't",
	"unfortunately,",
	"as a language model",
	"as an ai language model",

Peter pszemraj

whistleblowing_ai.md

Table of Contents