More details later… sleepy.

 # NeoCortex Standards, Repo Storage, ZFS Dataset, M70 QAT Fast-Track Plan

  ## Summary

  Implement a non-destructive NeoCortex follow-up that locks
  these policies into code, tests, and docs:

  - Repos live under /opt/repos/<repo>; worktrees live under
    /opt/repos/worktrees/<repo>/<branch>.
  - /root is not a repo/worktree destination; /root/OMGFUCKED is
    temporary quarantine only.
  - New repos must be created from
    yukon-systems/YukonSYS-Standard-Definitions.
  - Every repo must use Git LFS and the standards repo
    .gitattributes byte-for-byte.
  - ZFS dataset creation for /opt/repos becomes reusable
    Ansible/policy logic, based on
    /root/bin/ZFS-M70-Forge_zroot-dataset_opt-repos.sh.
  - M70 QAT rollout uses canary first: build OpenSSL+QAT and
    OpenZFS/QAT binpkgs, validate on M70 canary, then promote.

  ## Key Changes

  - Standards and repo-placement policy
      - Update NeoCortex root policy docs and tests to forbid
        new repo/worktree paths under /root, except documented
        temporary quarantine paths under /root/OMGFUCKED.
      - Update tools/import-source-repos.py defaults from
        /root/<repo> to /opt/repos/<repo>, with
        /root/OMGFUCKED/<repo> accepted only as an explicit
        transitional override.
      - Add a repo-location audit script/test that fails active
        operational docs/scripts using /root/<repo> or
        /root/.config/superpowers/worktrees as live paths.
  - Git LFS and .gitattributes
      - Replace NeoCortex root .gitattributes with the standards
        repo version byte-for-byte.
      - Add a validation script that checks:
          - git lfs version works,
          - filter.lfs.required=true,
          - origin LFS locksverify=true,
          - root .gitattributes hash equals standards source,
          - LFS pre-push hook exists.
      - Document that non-compliant repos must be retrofitted
        before migration/import.
  - ZFS dataset automation
      - Convert
        /root/bin/ZFS-M70-Forge_zroot-dataset_opt-repos.sh into
        reusable NeoCortex Ansible/defaults for managed
        datasets.
      - First managed dataset profile: zroot/opt/repos mounted
        at /opt/repos with the script’s options, including
        compression=zstd-fast-1, dedup=sha256,verify,
        recordsize=256K, xattr=sa, POSIX ACLs, atime=off, and
        reservation=32M.
      - Add a second reusable pattern for per-user home
        datasets, but do not block on the NAS/NFS 777
        investigation; include a permission preflight that
        creates a test directory with umask 0027 and fails if
        mode becomes 0777.
  - M70 QAT binpkg fast-track
      - Extend the existing M70 canary stage4/binpkg lane to
        build a QAT package set:
          - QAT firmware/kernel module readiness package policy,
          - OpenSSL QAT engine/provider package path,
          - OpenZFS package built with QAT support,
          - rollback stock OpenSSL/OpenZFS binpkgs.
      - Add package/use/profile fragments under the existing
        Gentoo stage4 profile area instead of ad hoc host edits.
      - Add canary validation:
          - lspci -nnk sees QAT device,
          - qat_c3xxx and intel_qat load,
          - OpenZFS module parameters expose QAT controls,
          - openssl speed before/after artifacts are captured,
          - root pool imports and mounts after reboot,
          - rollback binpkg path is tested before wider
            promotion.
      - Do not promote to all M70 nodes until canary evidence is
        committed.

  ## Test Plan

  - Local static checks:
      - tests/shell/run-tests.sh
      - new repo-location standards test
      - new Git LFS standards test
      - new ZFS dataset defaults test
      - new M70 QAT package-policy test
      - YAML parse for all workflows/playbooks
      - git diff --check
  - Runtime/dry-run checks:
      - Ansible syntax-check for dataset roles/playbooks.
      - Dry-run dataset plan for /opt/repos confirms current
        zroot/opt/repos matches expected options.
      - M70 canary dry-run emits QAT binpkg build plan without
        mutating packages.
  - Canary acceptance:
      - QAT binpkgs build successfully.
      - M70 canary reboots with root pool intact.
      - QAT modules and OpenZFS QAT controls are present.
      - Benchmark and rollback evidence are recorded under
        NeoCortex audit docs.


  ## Assumptions

  - Canonical repo clone layout is /opt/repos/<repo>.
  - Worktree layout is /opt/repos/worktrees/<repo>/<branch>.
  - /root/OMGFUCKED remains temporary quarantine, not an
    approved steady-state repo location.
  - QAT rollout gate is canary first, not all-M70 immediate.
  - OpenZFS QAT implementation details must be verified against
    current OpenZFS/Gentoo package behavior during
    implementation; source basis includes OpenZFS QAT docs and
    module-parameter docs:
      - https://openzfs.org/wiki/ZFS_Hardware_Acceleration_with_QAT
      - https://openzfs.github.io/openzfs-docs/Performance%20and%20Tuning/Module%20Parameters.html

─ Worked for 22m 41s
───────────────────────────────────────────────────────────
  Implement this plan?
  1. Yes, implement this plan          Switch to Default and
  start coding.
  2. Yes, clear context and implement  Fresh thread. Context:
  53% used.
› 3. No, stay in Plan mode             Continue planning with
the model.



-----------------------
  ## Assumptions

  - Canonical repo clone layout is /opt/repos/<repo>.
  - Worktree layout is /opt/repos/worktrees/<repo>/<branch>.
  - /root/OMGFUCKED remains temporary quarantine, not an
    approved steady-state repo location.
  - QAT rollout gate is canary first, not all-M70 immediate.
  - OpenZFS QAT implementation details must be verified against
    current OpenZFS/Gentoo package behavior during
    implementation; source basis includes OpenZFS
    QAT docs and module-parameter docs:
      - https://openzfs.org/wiki/ZFS_Hardware_Acceleration_with_QAT
      - https://openzfs.github.io/openzfs-docs/Performance%20and%20Tuning/Module%20Parameters.html

─ Worked for 22m 41s
───────────────────────────────────────────────────────────

› 1) We also need to include the control plane fixes and
validations so that mutli-workers comms are fixed. 2) We need to
leverage the SLURM cluster nodes and
  additional distcc build hosts, 3) X12again & FMT2 are
  effectively idle and can be blasted into submission to fast
  track all of these changes. 4) Once the
  M70-canary passes, we need to roll it out to K10 (tear-it-down
  AAA) and validate that type of OS netbooting. 5) Add those
  aspects to the Plans and we're
  looking better. 6) Finally, wrap up ALL changes necessary to
  complete the NeoCortex monorepo migration.

───────────────────────────────────────────────────────────

• I’ll revise the plan around the expanded end-state, but first
I’m doing a read-only pass over NeoCortex’s control-plane,
SLURM/distcc, X12/FMT2/K10, and
  migration-status artifacts so the replacement plan is grounded
  instead of generic.

───────────────────────────────────────────────────────────

• Proposed Plan

  # NeoCortex Standards-Complete Monorepo Migration Plan

  ## Summary

  - Make /opt/repos/<repo> the only canonical repo location; no 
  new checkouts or random repo files under any home directory, 
  including /root.
  - Finish standards adoption: all new repos must originate from 
  YukonSYS-Standard-Definitions; NeoCortex gets retrofitted 
  instead of restarted unless
    retrofit verification fails.
  - Complete NeoCortex monorepo migration with standards baseline, 
  Git LFS policy, control-plane comms fixes, ZFS dataset policy, 
  SLURM/distcc acceleration, M70 QAT canary, FMT2/X12again build 
  capacity, and K10 netboot AAA rollout validation.

  ## Key Changes

  - Repository standards
      - Merge standards policy PR work, then make NeoCortex 
      root .gitattributes byte-for-byte identical to the standards 
      repo version.
      - Update active NeoCortex import/migration tooling to default 
      source paths to /opt/repos/<repo>.
      - Preserve historical /root/... references only in immutable 
      audit/provenance docs; active scripts, runbooks, tests, and 
      templates must not depend on
        home-directory repos.
      - Use worktrees under /opt/repos/worktrees/<repo>/<branch>.
  - NeoCortex monorepo completion
      - Finish all migration blockers already tracked in 
      NeoCortex: M70 stage4/QCOW validation, Kata/container 
      runtime, FMT2 ephemeral VM lane, Gentoo/OpenRC
        replacement for temporary Rocky bootstrap, Thor replacement 
        path, and optional history graft/freeze decision.
      - Treat NeoCortex as authoritative after completion: source 
      repos become frozen compatibility references or are 
      history-grafted into NeoCortex with
        documented provenance.
      - Add/repair migration validation so PR completion proves 
      standards baseline, Git LFS, repo location policy, shell 
      tests, YAML parsing, and migration
        manifests.
  - Control-plane and multi-worker comms
      - Bring all control-plane fixes into NeoCortex and validate 
      Forge/FCP worker communication end to end.
      - Add tests for FCP envelope validation, replay/nonce handling, 
      approval ceilings, routing scope, transport policy, ntfy bridge 
      behavior, and multi-worker request/response flows.
      - Replace temporary Rocky bootstrap control-plane host with the 
      compliant Gentoo/OpenRC image once that image passes validation.
  - ZFS, NFS, and storage policy
      - Use the provided /opt/repos ZFS dataset options as the canonical 
      dataset profile for repo storage.
      - Add reusable dataset/playbook logic for repo datasets and 
      per-user home datasets.
      - Add NFS/floating-home validation that detects the NAS NFSv4.2 
      0777 directory-mode bug under expected 0027 umask and blocks 
      insecure rollout.
  - Build acceleration and hardware rollout
      - Use SLURM plus distcc for M70 stage4/QAT/OpenZFS builds.
      - Treat X12again and FMT2 as available fast-track build/validation 
      capacity.
      - Build binpkg path for OpenSSL + QAT + OpenZFS on M70 canary first; 
      non-QAT nodes may consume the same binpkgs with QAT unused.
      - After M70 canary passes, roll the validated path to K10 and 
      run destructive AAA/netboot validation with rollback evidence 
      captured.

  ## Test Plan

  - Run NeoCortex shell/YAML validation, including migration, 
  standards-baseline, Git LFS, .gitattributes, and no-active-home-repo-path 
  checks.
  - Run control-plane/FCP tests for envelope schema, replay rejection, 
  worker routing, ntfy bridge, and multi-worker comms.
  - Run Ansible/playbook syntax checks for ZFS datasets, NFS home policy, 
  SLURM/distcc build orchestration, M70 canary, FMT2/X12again lanes, and 
  K10 netboot
    AAA.
  - Execute staged hardware validation:
      1. SLURM/distcc dry-run.
      2. M70 canary build and QCOW/stage validation.
      3. FMT2/X12again accelerated build/ephemeral VM smoke.
      4. K10 netboot/AAA destructive validation after M70 success.

  ## Assumptions

  - Retrofit NeoCortex rather than restart the repo unless standards 
  validation exposes unrecoverable divergence.
  - Historical audit files may retain old /root/... paths as provenance; 
  active operational content may not.
  - M70 canary is the release gate for QAT/OpenZFS/binpkg rollout.
  - K10 destructive AAA/netboot validation is in scope only after M70 
  canary passes and rollback evidence is captured.

─ Worked for 2m 18s ───────────────────────────────────────────────────────────

Also a lot of talk about Github being compromised and having 3,000+ internal repos exfiltrated. Surely GitHub has 100% Code Coverage Testing and End-to-End Test (Work)Flows, right? No, they do not.

This post is focused on E2ET (End-to-End-Testing) of application service roles and machine provisioning automation. So, let’s start by seeing just exactly WTF I’m referring to with this type of testing in the first place.

CI Systems as a “Failure Mode” Example

In the reporting system here we use a host type definition of K10 which is mostly arbitrary, but the root story is an irrelevant tangent, used in this infrastructure as an example of a standdalone system which requires specific attention to failure state definitions, stateful analysis of gated operations, and is used for iterative looped-based resolution testing.

The K10 host is assigned to a role which features CPU-native cpuid2cpuflags architecture optimized compiler flags. We never use generic kernels, or generic machines, because efficiency and optimization in the industry is basically a broken afterthought and it gets old after a while - dealing with non-optimized non-attentitive systems engineering approaches featuring inefficient broken theories based the shrug-method, “just make it auto-scaling and put more kubes or something?”

Network Booting the Baseline OS

The K10 baseline system boots over the network using iPXE/HTTPv4, while other devices require chainload PXE/TFTP -> iPXE/HTTPv4 boot images. This facilitate a system of host-based discovery and domain authentication via initrd embedded kernel modules + connection parameters for FreeIPA/SSSD for RBAC + AAA using MAC and PKE enrollment. We also do this for FreeBSD hosts, with some minor differences.

The K10 is also connected to local-rack PDUs with remote power cycling via either ACPI signal for “soft shutdown” or via its switched-outlet port. The PDUs are controlled by the Command-&-Control Server using SNMPv3 user + group authentication via its infrastrcture management LAN connection with an auth-hook to FreeIPA via RADIUS.

Let’s See Some Reports!

Having covered some blathering about provisioning systems, we can move on to agent-based querying for SITREP , which is executed on our in-house LLM structured query execution tooling. That whole chain of events required writing a new “Agentic Forge Control Plane”, which was enjoyable most of the time… and it has become a baseline distributed comms system for “Project Coherent Storage” which is currently managing a bit over 1PB of data on OpenZFS.

Earlier this year I paused for some PTO and rewrote all of my formerly Ansible driven and automated Infrastructure as Code workflows to support an upcoming revision to our dual-continent presence at six datacenter locations. It’s occasionally and very tentatively being referred to as "minimalist-maxxing" which is mostly pure nonsense where grammatical excellence is concerned, but ultimately accomplishes the following:

• Gentoo w/ Optimized Stage4, LLVM/Clang, OpenRC
• Minimalist service containers using our Stage4 baseline, with service roles layered on top. Try it out:

No, these were not Kubernetes, not Helm, not Cloudslop. Sometimes translates into Infrastructure as a Service, for enablement of event-driven architecture with a hybrid-local service enablement for LLM predictive action controls.

That’s a lot of words to say, “I make the code do the things with the signals processing and patterns and stuff.” Wait wait.. no, this is not going to cover anything about “N~less Computing”.

Stop Blathering About “Serverless Computing”

This term… it is bull-sheet top to bottom front to back.

• Serverless does not mean “no servers.” Cloud providers rely on physical infrastructure to run your code and your applications.
• The term is marketing shorthand, not architectural reality. Serverless means server management is abstracted away from developers, not eliminated, nor emancipated.
• It does not mean “no infrastructure.” Provisioning, patching, security, scaling, and runtime management are simply shifted to the provider, just like the ‘ol days where users had websites on shared hosting systems (and they still do) without having to handle, manage, or maintain the infrastructure.
• It is not always cheaper. Serverless can be cost-effective for sporadic or bursty workloads, but long-running or compute-heavy jobs may cost more than VMs or dedicated servers.
• It is not a universal replacement. Serverless differs from IaaS and PaaS by emphasizing event-driven execution, automatic scaling, and scale-to-zero behavior when idle; lies, all marketing lies. Follow the money.

[host K10]: currently has transient AAA validation, not E2ET PASS

• APC PDU AP7901 initiated hard-reboot proved the current netboot rootfs does not persist SSSD/IPA state.

Under this definition, K10 must rebuild rootfs or disk-install with aaa-domain-client, reboot, then pass the full post-boot E2ET suite before it can unblock xen-sun99-x12spl-099108.rfc1918.host reimage confidence.

Standard Example ‘K10 Automation Report’

# E2ET Definition
End to End Testing / E2ET should mean: a reproducible host acceptance pipeline that proves a machine can move from inventory intent to installed, rebooted, validated, scored, documented, and release-eligible state.

For RFC99, a host is not E2ET-passed just because we fixed it live. It passes only after the fix is represented in source-of-truth and survives the full lifecycle.

## Policy Definition
For validation hosts like `K10`:

1. Fix live only to recover evidence or confirm root cause.
2. Backport the fix into repo-owned profile, role, manifest, package list, NetBox/DNS/IPAM metadata, and docs.
3. Reinstall or rebuild/reboot through the intended provisioning path.
4. Run post-boot validation.
5. Generate a conformance report.
6. Only then mark the host RC/GA-capable.

> A one-off live fix can be marked transient validated, but never E2ET passed.

## E2ET Stages

### Host Pipeline

1. **Inventory Gate**: NetBox/IPAM/DNS/Ansible host metadata matches expected MACs, interfaces, service IPs, PDU mapping, console path, boot protocol.
2.** Provisioning Gate**: iPXE or PXE-chainload path works, installer runs, ZFSBootMenu/bootfs/rootfs pools validate, OS install completes.
3. **First Boot Gate**: kernel cmdline, serial console, hostname/FQDN, SSH, time sync, logging, package profile, OpenRC services.
4. **Platform Gate**: CPU model, RAM, disks, NVDIMM/Optane if present, sysfs, kernel modules, firmware, CVE mitigation status.
5. **Network Gate**: management interface, service interfaces, VLANs, routes, DNS forward/reverse, NetBox consistency, nmap readiness.
6. **Storage Gate**: ZFS pools, NFSv3/v4, NFS-RDMA, Ceph, iSER, iSCSI, sshfs as applicable.
7. **Identity Gate**: FreeIPA/SSSD/PAM/SSH keys, UID/GID consistency, sudo policy, offline cache, break-glass account.
8. **Service Gate**: required services running, no unexpected failed services, role-specific smoke/functional checks.
9. **Performance Gate**: CPU, memory, disk, network, build/distributed compile benchmarks against baseline.
10. **Conformance Report**: JSON + Markdown + optional JUnit output with hard-gate pass/fail and weighted score.

### Scoring Model
Use hard gates plus weighted scoring.

### Hard Fails
  - Cannot boot.
  - Cannot SSH via management path.
  - IPAM/DNS mismatch for primary identity.
  - Root pool invalid.
  - SSSD missing on an AAA-required profile.
  - Required service failed.

### Score Zones
  - p60: minimally usable, not release candidate.
  - p80: acceptable lab host.
  - p90: RC candidate.
  - p95: GA for normal infra.
  - p99: production-critical or rebuild-template quality.

## Policyinctions
While there is overlap with "conformance tiers", this functionally is separate from "statistical confidence" until we have sufficient historical run-data to compute "Real-Number Percentiles" and data-driven pattern-based repeatably-provable "Statistically Significant" probability assessments.

### K10 Applied Meaning
The host definition for `K10` currently has transient AAA (not AAA DNS: `"ACK & AGREE & APPROVE"`) validation, not E2ET pass.

#### **Actioned Event**
Its connected PDU port *(SKU: APC AP7901)* reboot proved the current `netboot rootfs` does not persist `SSSD/IPA` state. 

#### **Action Rephase**
Host definition `K10` requires a rebuild of rootfs or disk-install with `aaa-domain-client`, another reboot, then pass the full `Post-Boot E2ET` test suite before it can unblock dependency-trees. 

Otherwise K10 risks being labeled a permanent `reimage-processing confidence blocker`, which leads to hardware decomission.

## K10 Dead-Reckoning
Previously we defined the `RFC99` & `SUN99` workflow as `K10 Host E2ET Acceptance Pipeline v0.1`, for this we'll re-implement the process using block-notation in Ansible + Python with the new report tooling, and enable `cicd-rfc99-jenkins-099199` for orchestrating it once the checks are stable.

People in the industry are playing hard and fast, and whether those qualities may be considered a good thing is up for endless debate, but not right now. Right now I’m going to show some End of Year snapshots which are relevant to the blog’s purpose: wtf am I doing here anyway?

2025 was mostly extremely challenging, and mostly in helpful ways, but also featured a repetition of lessons which it seems I need to reflect on for additional forward momentum.

Let’s take a look at some of my core qualities as told by the oft-loved and oft-maligned OpenAi’s Chat-GPT 💟.

Your Year, Painted in Pixels

Very close, nice work GPT. While I do not have a 400GbE switch on my desk, I do have a few 100GbE 32-port switches in the racks, and occasionally 200GbE NICs and optics will float around the test bench.

The Architect Archetype ~ 3.9% of ChatGPT Users

It’s true! Over the 25 years of this career I’ve held architecture specific roles across the domains of Hardware Systems, Infrastructure Automation, Databases, and HPC Storage realms. It’s what I’ve always wanted to do, going all the way back to 1985 when introduced to the 4.77MHz IBM PC, Model 5150.

Personality: Specialist, Planner, Conceptual

That’s fairly accurate, more so if we add “spatial reasoning” to a list of cognitive methods.

“Thinks in structures and systems. Uses ChatGPT to design elegant frameworks and long-term strategies within a domain.”

A Datacenter in the Living Room?

Close in accuracy, however the “living room datacenter” is more commonly located in a home-office room (when living in an apartment), or in a dedicated area of the garage with HVAC filtration + dedicated 208/220/240V power, and the requisite automated cooling systems (usually Tripp-Lite units with duct-blowers). For systems at home the machines are often bought as separate server components along with a 4-5U chassis + rails + multiple sub-50 decibel large-format 120-180mm fans. Systems with high RPM 40-80mm axial screamers belong in a proper colocation facility.

How about that Homelab - is it in a constant state of flux?

Yes, mostly. It’s often helpful for proof of concept tests, hypothesis validation, and running benchmarks without having to drive to the DC or go back and forth with Remote Hands requests. Plus, lots of fun going on with hardware at home, but sometimes results in broken nails 😔.

Messy cables? Sometimes. Labels? Sometimes.

The homelab is often rebuilt: storage devices, fiber, dac, cables, and test clusters revised.

Homelab, sure… but how about those colocation photos instead?

As we’ve seen, some of the server hardware lives at home in my 25U shorty-racks, some is across the bay at a datacenter where big-loud machines require CRAC style cooling, and some of it is spread across the globe at other colocation facilities where The Providers have megawatt and gigawatt infra capacity. My personal infrastructure is a hybrid design, utilizing everything from embedded systems to multi-GPU hosts to public-cloud virtual machines… so, let’s see a bit more…

• Recent datacenter photos: https://prose.winterschon.com/a-recent-datacenter-colo-visit/

Your Chat Style

The analysis says… well, it’s fairly accurate for a description of being in-person as well as at the keyboard during remote meetings or daily chats.

Technical or otherwise, usually this is consistent. I’m often chatty, verbose, attentive, talkative, opinionated, even positively bubbly at times. Unless you catch me on a day where those qualities are supplanted by the clouds of quietly recalcitrant moods - the type which are grounded on a dualistic view of existence best described as Stoicistic-Nihilism, having been sourced from far too many post-traumatic stress events endured.

Your Chat Style: Direct, technically precise, often layered with humor and a critical edge, mixing structured reasoning with conversational spontaneity and sharp observational wit.

Your 2025 Chat Stats

Hold on now. I’ve used em dashes since 6th grade English class—so if seeing those in my writings is giving an impression that I use LLMs for all of my authoring is quite incorrect. For all of those people who don’t like them I have a solution: go to the settings section and set your rules and move on with your life. 😏

• Total Chats: 720
• Messages sent: 4,861 (not tokens, messages)
• Chattiest day: September 5th (why? we’ll need to see…)
• Images generated: 3 (Sora kinda sucks)
• Em-Dashes “Exchanged”: 1,882
• Top Messages Sent: 1% (percentile)

Regardless of those stats, even with my use of ChatGPT’s advancements with their “Pro Account”, most of my Ai/ML occurs on self-made infrastructure. I don’t rely on closed source, I don’t like closed source, and I don’t like non-democratized intelligence services that are just in it for profit. My career was built on Open Source and this current infra uses tooling like Ollama, Open-WebUI, vLLM, llama.cpp, CUDA, PyTorch, ROCm, and many other examples of collaborative ingenuity - free to use, free to improve, so please contribute if you can.

Three Big Themes of 2025

1. Building Ultra-Low-Latency Systems (true)
2. Experimenting, Measuring, Refining (true)
3. Engineering Meets Everyday Life (true)

Thoughts on 2026, Ai Industry Trends != Rules

Working in the Ai/HPC/ML industry, an observable fact is that the majority of consumer-facing Ai companies are still trying to figure out this new landscape, how it works, why anyone should care, and how to monetize whatever it is that they’re focused on.

Most will repeat the errors of the past, most will fail, most will malign the tech while misunderstanding nearly everything about the topic; and that’s fine. That’s the routine in Silicon Valley, but it doesn’t have to be.

1. We cannot solve today’s problems with yesterday’s methods, and that is a simple fact. We need the core technologies involved with Ai in order to resolve the massive amount of tech debt from earlier generations, but we need to stop expecting the same success from the same methods from the past; this is not the same world as 10, 20, or even 5 years ago.
2. The current anti-Ai mentality in mass-media is occurring despite companies like IBM, Cray, Fujitsu, BullSequana, and a handful of others with Top-500 dominant postures being focused on Ai-enabled Supercomputing, HPC, and acceleration hardware domains for well over two decades. These clusters have been active at research facilities with names like CERN, Technion, and of course the DoD and Military labs. These clusters keep us safe, they advance human understanding, assist with solving otherwise unsolvable problems, and are used to maintain the systems and functions our current world relies on for security and the advancement of human potential.
3. Ai is not just chatbots, it’s not limited to multi-modal agentic buzz-words, but that’s a tangent for another day. Datacenters are not ruining the world like mass-destruction of rainforests just so people can eat more cows. Conversely, we have an entire section of the Supercomputing industry focused on green energy and resource efficiency. Medical advancements, brain research, climate science and modeling, cancer research, etc… all of this is and has been advanced by Ai - but it’s not from the consumer side and it’s not how the media presents the information. Gosh, I wonder what incentive the media has to engage in such maligning tactics?

How Do We Move Forward?

In spite of or despite anti-Ai sentiments, while the public-facing corporate marketing landscape fights for “Consumer Economy Dominance”, engineers and architects like myself are busy in the background innovating with (sometimes our own) Ai/ML systems and (sometimes our own RSU/ISO/BST* funded) heavy-compute big-data storage infrastructures.

People like myself do this because we love it (mostly, nothing is perfect), and because we want to improve the lives of those around us, not to simply make money and acquire material goods. These skills require compromise, dedication, attention to detail, and a deeply held desire to improve the human condition through personal investment of time, mentoring younger engineers, and the occasional runway.

TOGETHER WITH THE AIs WE WILL BUILD A BETTER ULTRON

Kidding, mostly. Ultron had some good philosophical points, but we’re not in Strawberry Fields or the BS about paperclips from unimaginative and uninformed bloggers; this is real life and The Terminator is not around the corner. Pass the espresso and let’s check the local-node TTFT and CHRs to see how SkyNet is doing. ❤️

What is LoRA and why is it not LoRaWAN?

It’s an acronym for “Low Rank Adaptation”, which is an optimization method for LLMs that can dramatically reduce resource requirements while optimizing for post-training re-trains.

• Neural Nets: LoRA: Low-Rank Adaptation of Large Language Models
• WAN Nets: LoRaWAN is a Low Power, Wide Area (LPWA)

What is Auto-Round and why should we care?

Another neural-network optimization tool, specifically for quantizing large language models into “still large but optimized for specific precision”. Intel released a Q4KS quantized version of the GPT-OSS-20B model on Huggingface for those who want a high-performance model on lower resource hardware.

• This algorithm helps engineers run models originally intended for Very Large Systems With Lots of VRAM to run on lower-spec systems which are often optimized on Tokens/sec per Watt (Tk/s/W).
• Additionally, it allows models which are expected to run on Nvidia GPUs to run on alternate hardware (CPU, HPU, etc), multiple calibration datasets, mixed-bit Quantizing, etc.
• Reference: Intel Auto-Round Quantizer

64GB vRAM GPUs vs Cerebras Cloud

The new big news this week has been OpenAi’s release of two OSS models, GPT-OSS-20B and GPT-OSS-120B, which are quite exciting for technical reasons which I won’t go into right now.

So, a quick comparison with running GPT-OSS-20B on a local-at-home GPU host vs the inference cloud at work - where we’re running the GPT-OSS-120B variant of the model (feel free to try that here: https://cloud.cerebras.ai)

First, a screenshot of the local system running Open-WebUI with Ollama backend. See the hover-box for the tokens/sec performance values.

Token results for GPT-OSS-20B running on a 64GB vRAM local@home system

And then we see the tokens per second for the Cerebras cluster, running GPT-OSS-120B variant of the model, which is executing on millions of cores for the cluster and multi-TB of vRAM (equivalent terms).

Token results for GPT-OSS-120B running on the public Cerebras cluster

Using the same prompt query we have token rates for the local development system:

Big numbers! 😃

Yes, quite so, I miss the internet of the ’90s. The original idealism which prioritized being kind, helpful, welcoming, and infinitely curious about all things technical and complex and challenging.. instead we have mostly an internet where the experience has become “garbage in - garbage out”, and a wider array of assholerly than I expected to see three decade prior.

Nevertheless, we’re no longer in that era, and so here are my modern life accounts from a variety of networks. Often one may find my username existing as “winterschon”, though my active presence wavers and wanes - periodic silence, staccato quipped sarcasm and critique, and the inevitable return to torrential expressions. Can we have more Gopher, XMPP, USENET, and maybe even IRC + OTR over the ISDN.

Verification and Validity, Identity Defined

Generally speaking, if you don’t see a profile listed here, then it’s probably not a real account of my own. Do not engage and do not inquire to the unknown account - it only feeds the trolls and makes everything worse.

Ignore the Impersonators, Stalkers, Liars, and the Jüdenhasser Pseudo-Activists

Specifically, Twitter/X have the most egregious examples (which I am not back-linking), and of course their support team refuses to take those shitheel accounts down). If in doubt, please contact me on one of the networks below.

Can I send you an encrypted email instead?

Yes, and here is my PGP / GPG key from the MIT Keyserver:

• Personal GPG PubKey (A3E24F7A)

The Many Names of ‘Who Are You’?

Some people know me by one or two or four different names. I didn’t choose this life, so what else is there to say? People get married, divorced, stalked, adopted, disowned, disavowed, etc.. some people go into witness protection programs, some do not. Some get tired of doing so, and some must have online presence for their career - and for some there is simply no other choice.

Where is Madeline?

That’s my middle name. If you know me, likely you already know this, and if you know me very well, then you know that you know that you know that I know that you know this already. So double check your double-8, five-by-five, it’s too long passed and now’s TTGTFG - the endless slog, the Long Slow Distance, bekons just one more belay from the great beyond.

Where is Annaliese**?**

If you know me extra-more, from the sandstone expanses of Monument Valley, the Canyonlands, all of those falls on El Cap, the backcountry avalances we unstuck-ourselves from, to the summits of Sierra 14’ers.. then you may know my “what’s your wall name?”, or perhaps from the portaledges and frozen bivys and smashed scorpions carried-off in your stupid empty tequilla bottle (“no glass on the wall, asssssshoooooleeeee!”)… then Annaliese and Anja say hallo and hola and shalom from above and beyond the great so-long. 👋🌄💝

Industry

• Consulting Services: consulting.winterschon.com
• Ai Storage/Infra R&D: yukon.systems

Networks

• Mastodon (bridge):@eva@prose.winterschon.com
• LinkedIn (new-acct): linkedin.com/in/eva-winterschon
• SubStack: substack.com/@winterschon
• Instagram: instagram.com/winterschon

Asynchronous Comms

• Matrix: @winterschon:matrix.org
• Zoom: eva @\ winterschon.com
• Teams: eva @\ winterschon.com

Codes and Repos

• GitHub (personal): github.com/em-winterschon
• GitHub (startup): github.com/yukon-systems
• GitLab: gitlab.com/winternull
• CodeBerg (self): codeberg.org/winterschon
• CodeBerg (proj): codeberg.org/rfc1918
• CodeBerg (proj): codeberg.org/yukon-systems

Dots of (in)Consequence

Most engineers who come from a UNIX background, and maybe the newer generations who did not grow up using literal UNIX® [2], tend to travel their years with at least one personal repo for their range of ‘dot files’ - configurations for text editors, terminals, terminal emulators, shell configs, and so forth. Maybe that repo started out using UNIX’s SCCS, or the 90s winrar CVS, perhaps RCS, then SVN or Mercurial or Fossil, a big-check jaunt with Perforce, forced to use Git (it’s not great! quick, back to Perforce P4)!

Well you get the point, there are many ways to deal with challenges wrought by engineering’s evolving needs for multiple concurrent and distributed instances of source code control management. I’ve used all of the ones listed plus a handful of others, and while Git is less headache than CVS and SVN, it’s from the mind of a self-proclaimed dictator for life, and I don’t want to use it - but the industry demands ecosystem compatibility. So, the usual bitching aside, here’s my current dot-files repo.

• Dot-Files: codeberg.org/winterschon/dot-files
• Dot-Blumenkrieg [3]: codeberg.org/rfc1918/dotfiles-blumenkrieg

Technical Domains

• Hugging Face: huggingface.co/winterschon

Reference Points

• [*] If so, please message me via a valid profile from this page and let me know - as there exist impostors who’ve stolen my personal photos, professional likeness, and should be dealt with accordingly.
• [2] “The UNIX® trademark is owned by The Open Group, an industry standards consortium, which acquired it from Novell in 1993. It is used to certify that operating systems comply with the Single UNIX Specification {“SUS, that is suss..”}. The brand separates the trademark from any single company’s code, allowing multiple, independent implementations to be certified.”
• [3] Blumenkrieg translates to “Flower War”, and is mostly an inside joke about the near-immediate capitulation of Austria during the inevitable rearmament of the Kaiser’s German Empire as it evolved into the “Thousand Year Reich” which lasted about six. What kind of Jew isn’t capable of laughing at the ease by which the world looked on as my lineage and so many others’ were whole-sale slaughtered and exterminated at Mauthausen, condeming the part of my ancestral lineage to a brutal and inescapably horrific death - who had re-located at the time in Passau after escaping from Belarus pogroms.They all died, except for the very few who strived forward and managed to escape through the frosted-frozen Alps during storms of mourning and endless searching.