====== Python for Scripting ======

===== Setup =====

<code bash>
python3 --version              # already installed
sudo apt install python3-pip python3-venv
</code>

Ubuntu blocks ''pip install'' system-wide (PEP 668): use a venv:

<code bash>
python3 -m venv ~/.venvs/myproj
source ~/.venvs/myproj/bin/activate
pip install requests
deactivate
</code>

Or install distro packages: ''sudo apt install python3-requests python3-bs4''.
Quick escape hatch without a venv: ''pip install --user --break-system-packages pkg''
:  fine for a personal laptop, worth knowing it bypasses the guard.

===== Script skeleton =====

<code python>
#!/usr/bin/env python3
"""one-line description"""
import sys

def main() -> int:
    return 0

if __name__ == "__main__":
    sys.exit(main())
</code>

chmod +x, ''./script.py''. Or a ''#!/usr/bin/env -S python3'' shebang.

===== Recipes =====

<code python>
# argparse
import argparse
p = argparse.ArgumentParser()
p.add_argument("file"); p.add_argument("-n", "--dry-run", action="store_true")
args = p.parse_args()

# files
from pathlib import Path
for f in Path("~/notes").expanduser().rglob("*.md"):
    text = f.read_text(encoding="utf-8")

# json / csv / yaml
import json, csv
data = json.loads(Path("in.json").read_text())
rows = list(csv.DictReader(open("in.csv")))
# yaml: pip install pyyaml -> yaml.safe_load(text)

# subprocess
import subprocess
out = subprocess.run(["grep", "-c", "x", "f.txt"],
                     capture_output=True, text=True, check=True).stdout

# http (stdlib, no deps)
from urllib.request import urlopen
html = urlopen("https://example.com", timeout=10).read()

# http (requests, if installed)
import requests
r = requests.get("https://api.github.com", timeout=10)
r.raise_for_status(); j = r.json()

# env + exit codes
import os, sys
token = os.environ.get("TOKEN") or sys.exit("set TOKEN first")

# dates
from datetime import datetime, timedelta
stamp = datetime.now().strftime("%Y-%m-%d %H:%M")
</code>

===== REPL & debugging =====

<code bash>
python3                     # repl
python3 -i script.py        # run then drop into repl
python3 -m pdb script.py    # debugger
breakpoint()                # in code, python >=3.7
</code>

===== Useful stdlib modules (no pip needed) =====

''pathlib'' ''shutil'' ''subprocess'' ''argparse'' ''json'' ''csv'' ''sqlite3'' ''re''
''datetime'' ''os''/''os.path'' ''sys'' ''textwrap'' ''logging'' ''http.server'' ''zipfile''
''tarfile'' ''statistics'' ''collections'' ''itertools'' ''functools'' ''dataclasses''

===== Quick servers & tools =====

<code bash>
python3 -m http.server 8000          # serve cwd over HTTP
python3 -m json.tool < file.json     # pretty-print/validate JSON
python3 -m venv --help
pip list / pip show pkg / pip install -r requirements.txt
</code>

===== When performance hurts =====

  * Startup is fine; CPU-bound loops are slow: use ''numpy'' for number work or
  push the hot path to ''subprocess''-ed tools (grep, ffmpeg).
  * ''python3 -O'' drops asserts. ''cProfile'' for real profiling:
  ''python3 -m cProfile -s cumtime script.py | head -20''


===== See Also =====

  * [[terminal:scripting:start|Scripting Index]]
  * [[terminal:tools:start|CLI Tools]]
  * [[terminal:start|Return to terminal]]
  * [[start|Return to wiki home]]