I still primarily use Spotify and YouTube to listen to music because they’re so convenient. But they often just recycle songs I’ve already saved or promote popular artists. That’s great when I just want something predictable to put on, but it’s less useful for music exploration. Spotify assigns every artist to a genre, and its suggestions rarely leave the top one hundred artists of whatever genre Spotify has decided you’re currently playing. You can see this directly on Every Noise at Once, a site built from Spotify’s own genre data. Its page for downtempo only lists about one hundred artists:
Hundreds of thousands of artists have made music in this style, but if you start a Spotify radio or check the related artists for any of them, you will rarely find music outside the artists listed above.
Currently, there aren’t any good online alternatives to streaming platforms for music discovery. That’s why so many serious music lovers “crate dig,” sifting through hundreds of random records at a record store and buying the ones that look promising. This process can be time-consuming and expensive, and most albums are likely to be mediocre. Still, this technique offers a unique advantage over streaming services: it lets you explore the full range of published music, not just the limited selection that streaming services typically suggest.
Physically searching through record stores isn’t the only way to get this kind of range, though. My main approach is to go one step higher and search the internet for music collections from dedicated collectors. Finding a collector’s personal library is much better than stumbling upon a single good record, since it can offer hundreds of hand-picked songs from diverse sources. The main problem with this approach is that serious music collectors are rare and scattered across the internet, mixed in with millions of listeners whose playlists consist solely of streaming service recommendations.
How does one actually find these collectors in the vast landscape of the internet? The best way is to start with obscure, high-quality songs you already love. I go through my music library and select a few of my favorite songs that aren’t on streaming services (or have very few views). If I can find these in someone else’s library, it’s a sign they actively seek out music from diverse channels, such as DJ mixes or small record labels. But most websites won’t let you search for user profiles by song. The only way I have found to find these profiles is through indirect methods.
In this series, I’ll break down my strategies for finding and filtering music collections. Some of these strategies are simple enough to try yourself in a few minutes with free online tools. Others are more complicated, and involve high-performance algorithms I designed, data science techniques and sifting through hundreds of gigabytes of public datasets. I even have a technique that uses metadata from P2P music piracy sites (just querying file lists, never downloading any music).
]]>Part one of this series showed how to verify code running in a cloud-hosted TDX TEE without trusting the cloud provider or deployer. This requires independently reconstructing four values (MRTD, RTMR0, RTMR1, and RTMR2) separately from the TEE and verifying that they match. RTMR1 and RTMR2 are derived from VM image components, such as the filesystem and kernel. MRTD and RTMR0 are more complicated because they depend on Google’s proprietary UEFI firmware. RTMR0 also depends on the machine’s initial environment, which changes with external factors like the amount of RAM allocated to the VM.
Most events in RTMR0 can be reproduced without analyzing the firmware: the five labeled “Hardware Description” are derived from metadata sent with the quote, and another six come from Secure Boot (which is redundant in TDX). Google regularly updates the default Secure Boot values, but they can be overridden with fixed ones during deployment. That leaves only two RTMR0 events (and MRTD) that change when Google updates their firmware. These must be derived from the compiled UEFI firmware file.
Fortunately, extracting useful data from a UEFI firmware file doesn’t require disassembling machine code. The files are organized as a tree of labeled regions rather than a single block of machine code. Reproducing these events doesn’t even require parsing the entire tree, since TDX-compatible firmware includes a metadata table listing every region used to build the VM’s initial memory.
To see how the table is used, it helps to know what “initial memory” means in TDX. A confidential VM’s memory is sealed off from the rest of the computer, so neither the hypervisor nor the cloud provider’s software can read or modify it. But something still has to set up the initial layout that the VM needs to start; the CPU performs this setup itself. It uses the TDX metadata table to allocate a portion of isolated memory to the confidential VM, recording each step it takes into MRTD.
The interactive diagram below shows the TDX metadata table from a recorded GCP TDX firmware file. Each entry contains a type field and a memory address range. Some entries also specify a range of bytes from the firmware file to copy into memory before starting the VM. Click any entry in the table at the top of the diagram to see where its data comes from and where it goes in the VM’s initial memory.
Three values change when Google updates their firmware: MRTD and the first two events of RTMR0. Two of these can be reproduced directly from the data in the table above.
As the CPU initializes the VM’s memory, it records each step into MRTD. This includes the memory ranges from every table entry and a hash of the BFV region’s file bytes. Since every TDX processor follows the same specification for this process, anyone can calculate the expected MRTD value using the data in the TDX metadata table.
The data hashed into MRTD does not include the CFV region’s contents, which is likely why Google modified their firmware to record it separately into RTMR0. Reproducing it just requires hashing the firmware region listed in the CFV row of the table.
That leaves the last event: the Hand-Off Block (HOB). Like MRTD, it is built from the regions in the metadata table. But while MRTD records only the initialized regions, the HOB also accounts for the memory between and above them, marking which parts of the VM’s memory start initialized and which are left for the OS to initialize later. And unlike MRTD, the HOB is not computed by the CPU. Instead, the hypervisor writes the HOB into the VM’s initial memory, then the firmware records a hash of its bytes into RTMR0.
Calculating the HOB is more involved than MRTD for two reasons. First, because it spans the VM’s entire memory range, including uninitialized parts, its contents vary with the VM’s RAM size. This is why Attest includes the RAM size with each quote. Second, the HOB is built by the hypervisor rather than copied from the firmware file. It just lists the regions from the metadata table, but the specification is loose enough that the exact bytes differ between hypervisors.
But Google’s hypervisor is proprietary, so there is no way to determine how it constructs the HOB. Patching Google’s firmware to print the HOB during boot isn’t an option either, since Google doesn’t allow running custom firmware. And the HOB is only used by the firmware, so its memory region is unallocated by the time the operating system starts. This means there is no direct way to read it from a running VM.
The data, however, remains in memory after it is unallocated. The TDX metadata table lists the HOB’s exact memory range, so I knew which address to read. I temporarily patched the Linux kernel to print the contents of that range to the kernel log as soon as the kernel boots. Running this across several VM sizes and comparing the results showed how Google’s hypervisor builds the HOB.
As shown in the interactive diagram below, Google’s hypervisor and the standard QEMU hypervisor build the HOB differently: they list the memory regions in different orders, and Google’s version includes several regions that QEMU omits. Toggle between the two hypervisors to compare their HOBs, and adjust the RAM size to see how the layout changes.
These layout rules are a property of the hypervisor, not the firmware, so they hold across firmware updates. Once they are known, the HOB can be reproduced for any GCP VM using only the metadata table and the RAM size.
With the HOB reproduced, every event in RTMR0 can be reconstructed from the firmware file and a few environment values, without needing to boot a VM or trust Google. This closes the last firmware-related gap in verifying a cloud-hosted TDX VM. The firmware, boot process, and VM image are now all checked against values computed by the verifier. Any change to them produces a different RTMR0, so a VM that has been tampered with fails verification.
Two inputs are still taken on trust: the ACPI tables and the Google firmware binary. The next posts in this series will explain how to handle these without compromising the trust model.
]]>Trusted Execution Environments (TEEs) let users send private data to third-party servers without trusting the server operator or cloud provider. For this to work, the data must stay private. TEEs can only guarantee this privacy if users verify the remote code that processes the data. However, that raises a basic question: how can they verify the code?
Earlier TEE technologies, such as Intel SGX, achieve this by isolating a single program from the rest of the computer, including the operating system. This simplifies verification but breaks compatibility with most software since even simple programs typically depend on an operating system for networking and file access. More recent TEE technologies, such as Intel TDX, are much more flexible. These newer TEEs use Confidential VMs (CVMs) to run an entire operating system within a trusted environment. Programs can be packaged into compact, bootable Linux images, enabling almost any unmodified software to run securely in a TEE.
Despite the advantages of CVM-based TEEs, one major drawback undermines confidence in their security. CVMs are harder to verify than previous TEE technologies because the verifier must check the entire boot process rather than just a single program. In Intel TDX TEEs, boot integrity is guaranteed by recording each boot stage, from the initial firmware to the point where the first process runs in the operating system. These records are combined into a set of hashes called measurements. Then, once the VM finishes booting, the software running within it can produce cryptographically signed reports, called quotes, containing these measurements. Today, cloud-compatible systems for verifying these quotes still require trust in the cloud provider or server operator.
To properly verify a quote, the verifier must know which measurements to expect from the VM. But this requires independently simulating the boot process, factoring in not just the remote application but also the operating system, firmware, and environment that loaded it. Several tools simulate this boot process, but they support only the QEMU hypervisor and OVMF firmware. By contrast, cloud providers like Google use proprietary hypervisors and firmware, which make the measurement process far harder to predict. Even the same VM image can produce dozens of valid measurements depending on environmental factors such as RAM size. Additionally, cloud providers regularly update their hypervisors and firmware, which change the measurements produced by newly deployed TDX images. These unpredictable changes make it hard for verifiers to reliably check measurements against expected values.
In practice, most deployments sidestep this problem by letting the cloud provider record the boot process and sign the measurements itself using software called a vTPM. But this undermines a core advantage of CVMs: eliminating the need to trust infrastructure operators. A user can only be fully confident that software in a CVM is safe to trust if they verify a quote directly from the CPU. To achieve this, they must independently simulate the boot process of a cloud-hosted CVM and compute its expected measurements. However, this requires software capable of precisely modeling the environment and boot process of cloud VMs, regardless of any updates to the cloud infrastructure.
To solve this problem, I built a tool called Attest, which can verify quotes even when the VM is running in the cloud. Attest uses the VM image file, firmware, and relevant environment details from the VM (such as available RAM) to compute expected measurements for both cloud and self-hosted environments. By allowing verifiers to independently generate and compare these measurements with those signed by the CPU, Attest enables trust in a VM’s integrity without requiring trust in the cloud provider.
This section walks through a real TDX boot process step by step and demonstrates how measurements are recorded and verified.
In TDX, a quote has two parts, both signed by a secret Intel key embedded in the CPU: the data (chosen by software running in the CVM), and several “measurement registers” that record the boot process. Each register is a protected storage slot in memory that stores a single hash. Every new piece of security-critical data is hashed with the register’s previous value to form a chain. Any change to this data alters the final register value, so any difference in the boot process can be detected.
Four TDX measurement registers are responsible for tracking the boot process: MRTD, RTMR0, RTMR1, and RTMR2. The CPU first reads the firmware that loads the VM and hashes it into MRTD. Next, the firmware records the VM’s initial environment into RTMR0. Subsequent steps in the boot process are recorded into RTMR1 or RTMR2, depending on which part of the process is being recorded.
The interactive diagram below illustrates the measurement sequence from a real Intel TDX VM instance. You can press play or scrub through the timeline to watch how the registers update at each stage. Clicking an event reveals detailed information about the data recorded at that stage.
All of these steps happen on the remote machine being verified. To check a quote, however, the verifier must independently reconstruct the expected register values for a trusted VM image and compare them to the quote signature produced by the CPU. Because each register is a hash chain, every boot event must be reproduced identically and in the same order.
The data needed to reproduce boot events comes from several sources. constant events are straightforward to verify because their values can be hardcoded. image events can be derived directly from the VM image. If the VM image is built reproducibly, anyone can rebuild it from source code and obtain the same image event hashes. This ties a TDX quote to the source code of the software that produced it. The environment events are the most complicated to reproduce since they depend on details like memory layout and hardware configuration, which vary across machines. The second post in this blog series covers how these details can ultimately be derived from a few basic machine details.
Only the RAM size, number of attached storage disks, and ACPI tables (which describe the attached hardware) are needed to reproduce RTMR0 for a given firmware. Attest’s proof-generation library packages these three values alongside each quote so the verifier can reliably reconstruct RTMR0 and verify the VM’s integrity. This lets the verifier validate every part of the VM’s environment except for dynamic elements that don’t need to be verified, such as the amount of available RAM or attached storage.
The interactive diagram below demonstrates the verification process. The simulation’s inputs are at the top. The left panel displays the VM’s reported environment details, while the right panel shows hashes calculated by rebuilding the VM image from source and hashing its components. Below the inputs, the main panel visualizes each boot event as it is reconstructed in sequence using the supplied inputs. At the bottom, you can observe how the rebuilt register values are checked against those in the signed quote.
Click the buttons next to each input to see how different changes affect verification. For example, clicking the filesystem toggle simulates a running VM instance whose filesystem differs from the published source code. This simulates a case where someone added a file or replaced a program with a backdoored copy before deploying. The boot process records this modified filesystem, so the signed quote reflects the change, but the verifier’s hashes are based on the original source code. When the two sides disagree, RTMR2 doesn’t match, and verification fails. Experiment with each input to see which registers are affected.
This protocol is implemented in the Attest GitHub repo as a Rust library and CLI tool for generating and verifying quotes. Nearly every input to the verification process can be reconstructed independently, except one: Attest does not attempt to rebuild the ACPI tables. Instead, it relies on the VM’s reported hashes. This is safe as long as the VM image is configured to process the ACPI tables in a sandbox, which is covered in part three of this series.
This work depended on understanding exactly what Google’s firmware records during boot, a process they don’t document publicly. Since Google doesn’t allow booting modified firmware and both their firmware and hypervisor are closed source, I had no way to inspect the underlying values hashed at each boot stage. Instead, I had to manually reverse-engineer these values. Now, Attest can extract all necessary values directly from Google’s firmware without simulating anything in a virtual machine. The next post in this series explains how this works.
]]>