Case Studies |

Exporting Local WeChat Voice Messages on macOS

Tue, 13 Oct 2026 00:00:00 +0000

A mate wanted to keep some WeChat voice messages as ordinary audio files and asked me to help. It sounded like a small conversion job: find the files, turn them into MP3s, and give them sensible names.

Finding the audio took more work than converting it. The Mac could have a message’s timestamp and other details without having the recording itself, and the files that looked promising at first weren’t usable SILK audio.

I ended up writing a Python script to read the recordings from WeChat’s local databases. It can export what’s on the Mac; it can’t recover audio that never made it there.

Table of Contents

Looking for the recordings

I started with a saved collection and some .silk_temp files. The decoder rejected those files as invalid SILK v3 data, so that route went nowhere.

What worked was reading VoiceInfo.voice_data from the local media_*.db databases. I could then match those audio bytes to the timestamps and IDs in the chat records.

The script therefore works with local chat and media records. It doesn’t extract resources held only in Favourites, copy messages from a phone or record playback.

Note

I used WeChat 4.1.15 on macOS, Python 3.14 and pilk 0.2.4. The code requires Python 3.10 or later, SQLCipher and FFmpeg/ffprobe. Other WeChat versions may need changes to the database queries or key-capture helper.

Reading the databases

On my Mac, the account directories were under:

~/Library/Containers/com.tencent.xinWeChat/Data/Documents/xwechat_files/
 <account>/db_storage/
 contact/contact.db
 message/message_0.db
 message/media_0.db
 ...

I scan only contact.db and numbered message_N.db / media_N.db files. The numeric suffix check excludes other databases with similar names.

These are encrypted databases. Python’s ordinary sqlite3.connect() cannot open them with a SQLCipher key. My small reader loads Homebrew’s libsqlcipher.dylib through ctypes, opens each file with SQLITE_OPEN_READONLY, supplies the raw key through sqlite3_key, selects SQLCipher 4 compatibility and enables query_only. A query against sqlite_master checks that the key actually works.

The reader returns query results as dictionaries. Its C callback receives text, so I request hex(voice_data) when retrieving a BLOB and convert it back with bytes.fromhex(). That avoids accidentally treating binary audio as a NUL-terminated text string.

The is at the end of the post, including the SQLCipher reader and its MIT notice. I’ll go through the queries and conversion code first.

Matching messages to audio

I use the contact database to resolve a name to a username, look up that conversation in the message databases, then find the corresponding audio in the media databases.

Each message database has its own Name2Id mapping. Reusing a numeric sender ID from another database can point to the wrong person, so the lookup must happen in the same database as the message. Audio matching also needs both the conversation and the server message ID.

In this schema, a conversation’s message table is named Msg_ followed by the MD5 digest of its username. I generate that name and query each message database:

import hashlib

table = "Msg_" + hashlib.md5(username.encode("utf-8")).hexdigest()
message_sql = f"""
SELECT m.local_id, m.server_id, m.create_time,
 n.user_name AS sender
FROM {table} AS m
LEFT JOIN Name2Id AS n ON n.rowid = m.real_sender_id
WHERE (m.local_type & 4294967295) = 34
"""

The low 32 bits of local_type identify a voice message as 34 in this layout. The table name is generated from a fixed prefix and a hexadecimal digest, rather than interpolated from an arbitrary SQL identifier. I check sqlite_master before querying because a particular conversation need not appear in every shard.

The media lookup runs separately in each media shard, using that shard’s own name mapping:

SELECT v.rowid AS media_rowid, v.svr_id,
 length(v.voice_data) AS bytes
FROM VoiceInfo AS v
JOIN Name2Id AS n ON n.rowid = v.chat_name_id
WHERE n.user_name = :username;

:username denotes the resolved contact identifier. In the C-API wrapper included below, which exposes sqlite3_exec rather than bound statements, the caller uses a SQL string-literal helper that doubles single quotes. A Python DB-API implementation could bind this value instead.

I collect candidate rows by server ID after filtering them to the selected conversation. The core of that join is:

from collections import defaultdict


def match_audio(messages, media_rows):
 # media_rows already belong to this conversation, across all media shards.
 candidates = defaultdict(list)
 for row in media_rows:
 sid = str(row["svr_id"] or "0")
 if sid != "0" and int(row["bytes"] or 0) > 0:
 candidates[sid].append(row)

 for message in messages:
 matches = candidates.get(str(message["server_id"] or "0"), [])
 status = ("available" if len(matches) == 1 else
 "not_local" if not matches else "ambiguous_audio")
 yield {**message, "status": status,
 "audio": matches[0] if len(matches) == 1 else None}

This short version shows the matching rule. The exporter also carries the source shard and media row ID, deduplicates message records by non-zero server ID, and stops if duplicate message records disagree about timestamp or direction. A zero server ID cannot identify an audio match. For the default received-only filter, a message counts as received only when its resolved sender equals the contact’s username; an unresolved sender is not guessed.

Each record gets one of these statuses:

Status	Meaning
`available`	Exactly one non-empty local audio row matched the message.
`not_local`	No matching non-empty audio row was found.
`ambiguous_audio`	More than one candidate matched; export skips it.
`exported`	Conversion and the output checks completed for that record.

An available record still needs checking when it’s exported. The script reads the bytes again and checks their size and SILK header before decoding; changed or invalid data causes an error.

Database keys and live files

Listing and export use SQLCipher in read-only mode, without restarting WeChat or requesting a database checkpoint. Recent records may still be in SQLite’s write-ahead log (WAL), so copying just the live .db file can miss them. I read the database in place, with its sidecar files, and let SQLite handle the WAL.

The SQLCipher reader and capture helper are adapted from . I retained the MIT attribution and narrowed the helpers to contact, message and media databases, with a separate wrapper for temporary capture and cleanup.

Working keys can be reused. To obtain missing keys or replace invalid ones, I use a helper that requires native Apple Silicon macOS and Xcode Command Line Tools, with WeChat quit normally beforehand. It creates a temporary app copy, re-signs that copy and runs it under LLDB. Candidates are checked against database-page HMACs and real read-only SQLCipher queries before they are saved.

This leaves the installed app and SIP unchanged, but the temporary copy still uses the real account’s data. WeChat may update its caches or login state. I save drafts before quitting and avoid writing messages in the temporary copy.

Running the example

You’ll need existing, verified keys to run the example below. Key capture isn’t included in it; the public toolkit linked above documents the setup I adapted.

In that helper, the database’s first 16 bytes identify its salt. On the tested arm64 build, an LLDB breakpoint on CCKeyDerivationPBKDF lets the helper inspect derivations for those selected salts. It recognises the SHA-512 derivation with 256,000 rounds, and the two-round derivation used for the page HMAC key. I check each candidate against the first-page HMAC, then try a SQLCipher read before saving it. These checks depend on the WeChat and SQLCipher versions used here.

Put the verified database keys in a private JSON file. The reader accepts entries indexed by relative database path or salt. For the path format, replace each placeholder below with that database’s 64-character hexadecimal key:

{
 "keys": {
 "contact/contact.db": {"enc_key": "<64 hexadecimal characters>"},
 "message/message_0.db": {"enc_key": "<64 hexadecimal characters>"},
 "message/media_0.db": {"enc_key": "<64 hexadecimal characters>"}
 }
}

Include an entry for every numbered message and media database being scanned, not just shard zero. The loader rejects an ambiguous match and checks that the manifest has no group or other permissions. The key enters the C API in memory; it is not passed as a shell argument or printed in an error.

Copy the into voice_export.py in a new working directory, then install the dependencies in a virtual environment:

python3 -m venv .venv
.venv/bin/python -m pip --require-virtualenv install 'pilk==0.2.4'
brew install sqlcipher ffmpeg
chmod 600 /absolute/private/keys.json

List first, using your own database root and exact contact name or username:

.venv/bin/python voice_export.py \
 --db-root '/absolute/account/db_storage' \
 --key-file '/absolute/private/keys.json' \
 --contact 'exact username' \
 --since 2026-08-01 --until 2026-10-13

Repeat with --output '/absolute/path/to/new-export' --keep-silk to export and retain the original SILK. Leaving out --output only lists records. --direction received|sent|both and --timezone select direction and timezone.

List before exporting

I check the listing before exporting: the contact, message times, direction and available audio should all look right.

A nickname or remark may match more than one contact, so the script stops if it can’t resolve a unique username. It also requires a database root, which avoids picking the wrong account on a Mac with several logins.

It lists received voice messages by default. You can select sent messages or both directions, but group chats aren’t supported.

Date filtering uses Asia/Shanghai by default. Both boundary dates are inclusive: internally, the upper bound is midnight at the start of the following day. Using an explicit timezone keeps a message near midnight from moving unexpectedly between dates when the script runs in another environment.

Only after checking that listing do I export to a new directory. The script refuses an existing output path, even an empty directory. That makes retries a little less convenient, but avoids mixing a partial attempt with a fresh export or overwriting an earlier result.

Converting SILK to MP3

The conversion path is SILK → PCM → MP3. pilk decodes at 24 kHz, and FFmpeg reads the intermediate PCM as mono, signed 16-bit little-endian audio before encoding it with libmp3lame.

The extraction step rechecks the selected row’s conversation and server ID as well as its row ID. After bytes.fromhex(), it requires the same byte count observed during listing and one of the two recognised headers: b"#!SILK_V3" or b"\x02#!SILK_V3". Merely finding a non-empty BLOB is not sufficient.

Here’s the conversion function on its own. It takes SILK bytes and an output path, refuses to overwrite files, and returns the durations and hashes for the manifest:

import hashlib
import json
import subprocess
import tempfile
from pathlib import Path

import pilk


def convert_voice(data, destination):
 if not data.startswith((b"#!SILK_V3", b"\x02#!SILK_V3")):
 raise ValueError("Not recognised SILK audio")
 destination = Path(destination)
 if destination.exists():
 raise FileExistsError(destination)

 with tempfile.TemporaryDirectory(prefix="voice-convert-") as folder:
 silk = Path(folder) / "voice.silk"
 pcm = Path(folder) / "voice.pcm"
 silk.write_bytes(data)
 pilk.decode(str(silk), str(pcm), pcm_rate=24000)
 size = pcm.stat().st_size
 if size == 0 or size % 2:
 raise ValueError("Empty or incomplete 16-bit PCM")
 decoded_seconds = size / (24000 * 1 * 2)
 subprocess.run([
 "ffmpeg", "-v", "error", "-nostdin", "-n",
 "-f", "s16le", "-ar", "24000", "-ac", "1",
 "-i", str(pcm), "-c:a", "libmp3lame", "-q:a", "2",
 str(destination),
 ], check=True)

 subprocess.run([
 "ffmpeg", "-v", "error", "-nostdin", "-i", str(destination),
 "-f", "null", "-",
 ], check=True)
 probe = json.loads(subprocess.check_output([
 "ffprobe", "-v", "error", "-show_entries",
 "format=duration:stream=codec_name", "-of", "json",
 str(destination),
 ], text=True))
 if not probe["streams"] or probe["streams"][0]["codec_name"] != "mp3":
 raise ValueError("Output is not MP3")
 seconds = float(probe["format"]["duration"])
 if abs(seconds - decoded_seconds) > 0.25:
 raise ValueError("Unexpected duration change")
 return {
 "decoded_seconds": decoded_seconds,
 "mp3_seconds": seconds,
 "silk_sha256": hashlib.sha256(data).hexdigest(),
 "mp3_sha256": hashlib.sha256(destination.read_bytes()).hexdigest(),
 }

At 24,000 samples per second, one channel and two bytes per sample, the PCM uses 48,000 bytes per second; dividing its size by that gives the duration. FFmpeg’s -n flag refuses an existing output even if it appeared after the Python existence check.

After encoding, I decode the whole MP3 with FFmpeg and use ffprobe to check its codec and duration. A difference of more than 0.25 seconds from the PCM duration fails the check. This catches conversion problems, though I’d still listen to recordings that matter.

Filenames use the original message timestamp: YYYY-MM-DD_HH-MM-SS.mp3. I count the available messages per second before exporting and append the server message ID wherever timestamps collide. Temporary PCM is deleted automatically. The optional SILK copy keeps the original bytes, since MP3 is lossy.

The JSON and CSV manifests record identifiers, timestamps, source databases, durations, hashes and statuses. Missing or ambiguous audio remains visible there instead of disappearing from the totals.

After each successful conversion, I add the record to the JSON manifest, write it to a temporary file in the output directory and use os.replace() to replace the previous version. This avoids leaving half-written JSON if a write is interrupted. The audio files and final CSV are still written separately.

If export fails part-way through, the JSON manifest contains the records processed so far. It may omit the failed record and everything after it, and the CSV may not exist yet. Fix the cause and retry into a new directory; the presence of some MP3s doesn’t mean the whole job finished.

Keys, cleanup and testing

I keep keys in a .local/ directory with mode 0700, and keys.json has mode 0600. The example below lets you choose the location with --key-file. Take care when copying the working folder: the keys can go with it. I keep them, the manifests and the recordings out of version control.

My key-capture wrapper removes the temporary app after completion, an ordinary error or a handled cancellation. A crash or forced termination can leave it behind, so I also have a cleanup helper for marked capture directories. It refuses to remove a temporary app that’s still running.

I tested the implementation with synthetic encrypted databases and synthetic audio. The cases cover sender mappings, ambiguous contacts, missing or duplicate media, actual SILK-to-MP3 conversion and the key-refresh helper’s cleanup behaviour. That lets me exercise awkward cases without turning my mate’s recordings into test fixtures.

The script gives me both the audio files and a list of what it couldn’t export. That second part matters: otherwise, a missing recording is easy to overlook amongst the MP3s.

Complete example

Copy this block into voice_export.py. It includes the reader, message lookup, export code and command-line entry point, along with the reader’s MIT notice. You’ll need the verified key file described above.

#!/usr/bin/env python3
"""Export locally stored WeChat 4.x voice messages on macOS.

Requires existing verified keys, SQLCipher, pilk and FFmpeg/ffprobe.
"""
# SQLCipher reader adapted from acmerfight:
# https://gist.github.com/acmerfight/0a01249ef72970d07a2603dbb629e80f
# The following notice covers the adapted reader.
# MIT License
#
# Copyright (c) 2026 Contributors
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.

import ctypes as C
import ctypes.util
import json
import os
from pathlib import Path
import re


def library():
 candidates = [os.environ.get('SQLCIPHER_LIBRARY'),
 '/opt/homebrew/opt/sqlcipher/lib/libsqlcipher.dylib',
 '/usr/local/opt/sqlcipher/lib/libsqlcipher.dylib',
 ctypes.util.find_library('sqlcipher')]
 name = next((x for x in candidates if x and (not x.startswith('/') or Path(x).exists())), None)
 if not name:
 raise RuntimeError('SQLCipher library missing; install with brew install sqlcipher')
 lib = C.CDLL(name)
 lib.sqlite3_open_v2.argtypes = [C.c_char_p, C.POINTER(C.c_void_p), C.c_int, C.c_char_p]
 lib.sqlite3_open_v2.restype = C.c_int
 lib.sqlite3_close.argtypes = [C.c_void_p]
 lib.sqlite3_key.argtypes = [C.c_void_p, C.c_void_p, C.c_int]
 lib.sqlite3_key.restype = C.c_int
 lib.sqlite3_exec.argtypes = [C.c_void_p, C.c_char_p, C.c_void_p, C.c_void_p, C.c_void_p]
 lib.sqlite3_exec.restype = C.c_int
 lib.sqlite3_busy_timeout.argtypes = [C.c_void_p, C.c_int]
 return lib


class DB:
 def __init__(self, path, key):
 if not re.fullmatch(r'[0-9a-fA-F]{64}', key):
 raise ValueError('expected 64 hex characters for raw SQLCipher key')
 self.lib = library()
 self.ptr = C.c_void_p()
 flags = 1 # SQLITE_OPEN_READONLY
 rc = self.lib.sqlite3_open_v2(os.fsencode(path), C.byref(self.ptr), flags, None)
 if rc:
 self.close()
 raise RuntimeError(f'SQLCipher open failed (code {rc})')
 try:
 key_arg = ("x'" + key + "'").encode('ascii')
 rc = self.lib.sqlite3_key(self.ptr, key_arg, len(key_arg))
 if rc:
 raise RuntimeError(f'SQLCipher key setup failed (code {rc})')
 self.query('PRAGMA cipher_compatibility=4;')
 self.lib.sqlite3_busy_timeout(self.ptr, 2000)
 self.query('PRAGMA query_only=ON;')
 self.query('SELECT count(*) FROM sqlite_master;')
 except BaseException:
 self.close()
 raise

 def query(self, sql):
 rows = []
 callback_type = C.CFUNCTYPE(C.c_int, C.c_void_p, C.c_int,
 C.POINTER(C.c_char_p), C.POINTER(C.c_char_p))
 def collect(_, n, values, names):
 rows.append({names[i].decode(): values[i].decode('utf-8', 'replace')
 if values[i] is not None else None for i in range(n)})
 return 0
 callback = callback_type(collect)
 rc = self.lib.sqlite3_exec(self.ptr, sql.encode(), callback, None, None)
 if rc:
 # No raw SQL or native error strings, since those may reveal secrets/content.
 raise RuntimeError(f'SQLCipher query failed (code {rc})')
 return rows

 def close(self):
 if self.ptr:
 self.lib.sqlite3_close(self.ptr)
 self.ptr = C.c_void_p()

 def __enter__(self):
 return self

 def __exit__(self, *args):
 self.close()


def load_key(path, db_path):
 p = Path(path)
 if p.stat().st_mode & 0o077:
 raise RuntimeError('key file must be private (chmod 600)')
 data = json.loads(p.read_text())
 if isinstance(data, dict) and isinstance(data.get('keys'), dict):
 data = data['keys']
 if not isinstance(data, dict):
 raise RuntimeError('key manifest must be an object')
 target = Path(db_path).resolve()
 with target.open('rb') as f:
 salt = f.read(16).hex()
 entries = []
 for name, value in data.items():
 normalized = name.replace('\\', '/')
 absolute_match = normalized.startswith('/') and Path(normalized).resolve() == target
 if normalized == salt or str(target).endswith('/' + normalized) or absolute_match:
 if isinstance(value, dict):
 value = value.get('enc_key', value.get('key'))
 if isinstance(value, str) and re.fullmatch('[0-9a-fA-F]{64}', value):
 entries.append(value.lower())
 if len(set(entries)) != 1:
 raise RuntimeError('missing or ambiguous key for this database')
 return entries[0]


import argparse
import collections
import csv
import hashlib
import json
import os
import re
import shutil
import subprocess
import sys
import tempfile
from datetime import date, datetime, time, timedelta
from pathlib import Path
from zoneinfo import ZoneInfo


def quote(value):
 return "'" + str(value).replace("'", "''") + "'"


def required_databases(root):
 return sorted([*(root / "contact").glob("contact.db"),
 *[p for p in (root / "message").glob("message_*.db")
 if re.fullmatch(r"message_\d+\.db", p.name)],
 *[p for p in (root / "message").glob("media_*.db")
 if re.fullmatch(r"media_\d+\.db", p.name)]])


def private_write(path, value):
 path.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
 fd, tmp = tempfile.mkstemp(prefix=".write-", dir=path.parent)
 try:
 with os.fdopen(fd, "w") as f:
 json.dump(value, f, ensure_ascii=False, indent=2)
 os.replace(tmp, path)
 finally:
 if os.path.exists(tmp):
 os.unlink(tmp)


class Reader:
 def __init__(self, root, keyfile):
 self.root, self.keyfile = Path(root), Path(keyfile)

 def open(self, path):
 try:
 key = load_key(self.keyfile, path)
 return DB(path, key)
 except (RuntimeError, OSError, ValueError) as e:
 raise ValueError("Cannot read " + str(path.relative_to(self.root)) +
 ". Supply a verified key manifest for this database; original error: " + str(e)) from e

 def contacts(self, query):
 with self.open(self.root / "contact/contact.db") as db:
 rows = db.query("SELECT username,alias,remark,nick_name FROM contact")
 needle = query.casefold()
 return [r for r in rows if not (r["username"] or "").endswith("@chatroom")
 and any(needle in (r[k] or "").casefold()
 for k in ("username", "alias", "remark", "nick_name"))]

 def resolve(self, query):
 matches = self.contacts(query)
 exact = [r for r in matches if any((r[k] or "").casefold() == query.casefold()
 for k in ("username", "alias", "remark", "nick_name"))]
 if len(exact) != 1:
 raise ValueError("Contact is absent or ambiguous. Run contacts and use the exact username.")
 return exact[0]

 def voices(self, contact, direction="received", since=None, until=None, timezone="Asia/Shanghai"):
 user = contact["username"]
 tz = ZoneInfo(timezone)
 lo = int(datetime.combine(date.fromisoformat(since), time.min, tz).timestamp()) if since else None
 hi = int(datetime.combine(date.fromisoformat(until) + timedelta(days=1), time.min, tz).timestamp()) if until else None
 if lo is not None and hi is not None and lo >= hi:
 raise ValueError("--since must not be later than --until")
 table = "Msg_" + hashlib.md5(user.encode()).hexdigest()
 messages = {}
 shards = [p for p in required_databases(self.root) if re.fullmatch(r"message_\d+\.db", p.name)]
 if not shards:
 raise ValueError("No local message shards found")
 for path in shards:
 with self.open(path) as db:
 if not db.query("SELECT name FROM sqlite_master WHERE name=" + quote(table)):
 continue
 rows = db.query("SELECT m.local_id,m.server_id,m.create_time,n.user_name AS sender "
 "FROM " + table + " m LEFT JOIN Name2Id n ON n.rowid=m.real_sender_id "
 "WHERE (m.local_type & 4294967295)=34")
 for row in rows:
 received = row["sender"] == user
 if direction == "received" and not received:
 continue
 if direction == "sent" and (received or row["sender"] is None):
 continue
 timestamp = int(row["create_time"])
 if (lo is not None and timestamp < lo) or (hi is not None and timestamp >= hi):
 continue
 sid = str(row["server_id"] or "0")
 ident = sid if sid != "0" else path.name + ":" + row["local_id"]
 record = dict(message_id=sid, local_id=row["local_id"], timestamp=timestamp,
 sent_at=datetime.fromtimestamp(timestamp, tz).isoformat(),
 direction="received" if received else ("sent" if row["sender"] else "unknown"),
 message_database=path.name)
 if ident in messages and any(messages[ident][k] != record[k]
 for k in ("timestamp", "direction")):
 raise ValueError("Conflicting message records for server ID " + sid)
 messages.setdefault(ident, record)
 media = collections.defaultdict(list)
 for path in required_databases(self.root):
 if not re.fullmatch(r"media_\d+\.db", path.name):
 continue
 with self.open(path) as db:
 rows = db.query("SELECT v.rowid AS media_rowid,v.svr_id,length(v.voice_data) AS bytes "
 "FROM VoiceInfo v JOIN Name2Id n ON n.rowid=v.chat_name_id "
 "WHERE n.user_name=" + quote(user))
 for row in rows:
 if row["svr_id"] and row["svr_id"] != "0" and int(row["bytes"] or "0") > 0:
 media[row["svr_id"]].append((path.name, row["media_rowid"], int(row["bytes"])))
 result = []
 for record in messages.values():
 matches = media.get(record["message_id"], [])
 record["available"] = len(matches) == 1
 record["status"] = "available" if len(matches) == 1 else ("not_local" if not matches else "ambiguous_audio")
 record["audio_bytes"] = matches[0][2] if len(matches) == 1 else None
 record["_audio"] = matches[0] if len(matches) == 1 else None
 result.append(record)
 return sorted(result, key=lambda r: (r["timestamp"], r["message_id"]))

 def audio(self, record, user):
 shard, rowid, size = record["_audio"]
 with self.open(self.root / "message" / shard) as db:
 rows = db.query("SELECT hex(v.voice_data) AS body FROM VoiceInfo v "
 "JOIN Name2Id n ON n.rowid=v.chat_name_id WHERE v.rowid=" + str(int(rowid)) +
 " AND v.svr_id=" + str(int(record["message_id"])) +
 " AND n.user_name=" + quote(user))
 if len(rows) != 1:
 raise ValueError("Audio changed since listing; rerun the command")
 data = bytes.fromhex(rows[0]["body"])
 if len(data) != size or not data.startswith((b"#!SILK_V3", b"\x02#!SILK_V3")):
 raise ValueError("Audio is incomplete or not recognised SILK")
 return data


def public(record):
 return {k: v for k, v in record.items() if not k.startswith("_")}


def export(reader, contact, rows, output, timezone, keep_silk=False):
 import pilk
 ffmpeg, ffprobe = shutil.which("ffmpeg"), shutil.which("ffprobe")
 if not ffmpeg or not ffprobe:
 raise ValueError("Install ffmpeg and ffprobe (brew install ffmpeg)")
 if not any(r["available"] for r in rows):
 raise ValueError("No unambiguous local audio to export")
 output = Path(output).expanduser().resolve()
 output.mkdir(parents=True, exist_ok=False, mode=0o700)
 counts = collections.Counter(r["timestamp"] for r in rows if r["available"])
 manifest = {"contact": contact["username"], "timezone": timezone,
 "source": "Local WeChat chat/media databases; no favourites or playback recovery",
 "records": []}
 private_write(output / "manifest.json", manifest)
 for row in rows:
 item = public(row)
 if row["available"]:
 data = reader.audio(row, contact["username"])
 stamp = datetime.fromtimestamp(row["timestamp"], ZoneInfo(timezone)).strftime("%Y-%m-%d_%H-%M-%S")
 if counts[row["timestamp"]] > 1:
 stamp += "_" + row["message_id"]
 mp3 = output / (stamp + ".mp3")
 with tempfile.TemporaryDirectory(prefix="wechat-voice-") as tmp:
 silk, pcm = Path(tmp) / "voice.silk", Path(tmp) / "voice.pcm"
 silk.write_bytes(data)
 pilk.decode(str(silk), str(pcm), pcm_rate=24000)
 if pcm.stat().st_size == 0 or pcm.stat().st_size % 2:
 raise ValueError("Decoder produced empty or invalid PCM")
 decoded = pcm.stat().st_size / 48000
 subprocess.run([ffmpeg, "-v", "error", "-nostdin", "-n", "-f", "s16le",
 "-ar", "24000", "-ac", "1", "-i", str(pcm),
 "-c:a", "libmp3lame", "-q:a", "2", str(mp3)], check=True)
 subprocess.run([ffmpeg, "-v", "error", "-nostdin", "-i", str(mp3),
 "-f", "null", "-"], check=True)
 probe = json.loads(subprocess.check_output([ffprobe, "-v", "error", "-show_entries",
 "format=duration:stream=codec_name", "-of", "json", str(mp3)], text=True))
 if not probe["streams"] or probe["streams"][0]["codec_name"] != "mp3":
 raise ValueError("Output is not MP3")
 duration = float(probe["format"]["duration"])
 if abs(duration - decoded) > 0.25:
 raise ValueError("MP3 duration differs unexpectedly from decoded audio")
 if keep_silk:
 (output / (stamp + ".silk")).write_bytes(data)
 item.update(filename=mp3.name, decoded_seconds=decoded, mp3_seconds=duration,
 silk_sha256=hashlib.sha256(data).hexdigest(),
 mp3_sha256=hashlib.sha256(mp3.read_bytes()).hexdigest(),
 status="exported")
 manifest["records"].append(item)
 private_write(output / "manifest.json", manifest)
 fields = sorted(set().union(*(r.keys() for r in manifest["records"])))
 with (output / "manifest.csv").open("w", newline="") as f:
 writer = csv.DictWriter(f, fieldnames=fields)
 writer.writeheader()
 writer.writerows(manifest["records"])
 return {"output": str(output),
 "exported": sum(r["status"] == "exported" for r in manifest["records"]),
 "skipped": sum(r["status"] != "exported" for r in manifest["records"])}


def main():
 os.umask(0o077)
 parser = argparse.ArgumentParser(description=__doc__)
 parser.add_argument("--db-root", type=Path, required=True)
 parser.add_argument("--key-file", type=Path, required=True)
 parser.add_argument("--contact", required=True, help="Exact username, nickname, remark or alias")
 parser.add_argument("--since")
 parser.add_argument("--until")
 parser.add_argument("--timezone", default="Asia/Shanghai")
 parser.add_argument("--direction", choices=("received", "sent", "both"), default="received")
 parser.add_argument("--output", help="Export into this NEW directory; omit to list")
 parser.add_argument("--keep-silk", action="store_true")
 args = parser.parse_args()
 root = args.db_root.expanduser().resolve()
 reader = Reader(root, args.key_file.expanduser().resolve())
 try:
 contact = reader.resolve(args.contact)
 rows = reader.voices(contact, args.direction, args.since, args.until, args.timezone)
 result = (export(reader, contact, rows, args.output, args.timezone, args.keep_silk)
 if args.output else {"contact": contact, "records": [public(r) for r in rows]})
 print(json.dumps(result, ensure_ascii=False, indent=2))
 except (ValueError, RuntimeError, OSError, subprocess.SubprocessError, ImportError) as error:
 parser.exit(1, "Error: " + str(error) + "\n")


if __name__ == "__main__":
 main()

What a RedNote Share Link Can Reveal

Tue, 13 Oct 2026 00:00:00 +0000

I came across on X that checks whether a shared link reveals the sharer’s social account. I was curious: when I send someone a post, what else goes with it?

I started by looking at the request the site sent when I clicked its query button. Calling that endpoint from Python would have been easy enough, but it wouldn’t explain how the lookup worked. I wanted to try the calculation myself, using one of my own links.

RedNote turned out to be a manageable example. I put together a small Python script for the token format I’d found; it needs only the standard library and works offline.

Table of Contents

The parameter to look for

A RedNote share URL can contain several IDs. The one in the /explore/ path belongs to the post; I was looking for the account behind the share.

The parameter I needed was shareRedId. I found the decoding method in cnlnn’s . That article describes a reversible transformation of a 24-character hexadecimal account ID:

Convert the ID to uppercase.
Add a fixed offset to each character’s ASCII value, using a different offset for each position.
Base64URL-encode the resulting 24 bytes.

To decode it, reverse those steps. There is no secret key involved: the fixed offsets are enough.

The method and the sample token below come from that article, under . I added input checks and a command-line interface.

Make the decoder fussy

I wanted malformed input to fail clearly, especially if RedNote changed the format later.

The script accepts a long RedNote URL, a Markdown link containing it, or a raw token. It normalises HTML-escaped ampersands and the escaped ampersands that can appear when copying Markdown. For URL input, it checks the hostname and requires exactly one non-empty shareRedId parameter.

Then it checks the token itself:

exactly 32 Base64URL characters;
decoded bytes which become exactly 24 uppercase hexadecimal characters after subtracting the offsets;
an exact match when those characters are encoded again.

The script returns the account ID in lowercase and builds a profile URL from it. Anything outside the supported format raises an error.

For a short link, you’ll need to follow its redirect first and copy the resulting long URL. The script doesn’t do that network request for you.

The Python script

Here is the complete script, including the calculation, input handling and command-line entry point. It uses only Python’s standard library. Copy this block into a local file named rednote_share_id.py to try it.

#!/usr/bin/env python3
"""Offline decoder for the known 24-byte RedNote shareRedId format.

Algorithm and synthetic vector: cnlnn, CC BY-NC-SA 4.0.
https://creativecommons.org/licenses/by-nc-sa/4.0/
Reference:
https://cnlnn.pages.dev/posts/share-link-identity-tracking/
No HTTP requests, third-party dependencies, or account enumeration.
"""
import argparse
import base64
import html
import json
import re
import sys
from urllib.parse import parse_qs, urlsplit

OFFSETS = (2, 6, 2, 0, 3, 5, 4, 9, 6, 7, 5, 2,
 9, 8, 0, 6, 6, 3, 9, 7, 4, 5, 6, 9)


def extract(value):
 """Decode one token or long share URL; reject unknown formats."""
 value = html.unescape(value.strip().replace(r"\&", "&"))
 urls = re.findall(r'https?://[^\s<>"\)]+', value)
 if urls:
 if len(urls) != 1:
 raise ValueError("Supply exactly one share URL.")
 parsed = urlsplit(urls[0])
 host = (parsed.hostname or "").lower()
 if host != "xiaohongshu.com" and not host.endswith(".xiaohongshu.com"):
 raise ValueError("Supply a RedNote long URL, not a short link or another domain.")
 params = parse_qs(parsed.query, keep_blank_values=True)
 candidates = params.get("shareRedId", [])
 if len(candidates) != 1 or not candidates[0]:
 raise ValueError("Expected one nonempty shareRedId parameter.")
 token = candidates[0]
 else:
 token = value

 if not re.fullmatch(r"[A-Za-z0-9_-]{32}", token):
 raise ValueError("Unsupported shareRedId: expected 32 Base64URL characters.")
 raw = base64.b64decode(token, altchars=b"-_", validate=True)
 recovered = bytes((byte - shift) % 256 for byte, shift in zip(raw, OFFSETS))
 if not re.fullmatch(rb"[0-9A-F]{24}", recovered):
 raise ValueError("Token does not match the known UID encoding.")
 uid = recovered.decode("ascii").lower()
 encoded = base64.urlsafe_b64encode(
 bytes(byte + shift for byte, shift in zip(recovered, OFFSETS))
 ).decode("ascii")
 if encoded != token:
 raise ValueError("Token failed the exact round-trip check.")
 return {
 "platform": "RedNote",
 "account_id": uid,
 "profile_url": "https://www.xiaohongshu.com/user/profile/" + uid,
 "evidence_parameter": "shareRedId",
 "round_trip_matches": True,
 "profile_verified": False,
 }


def main():
 parser = argparse.ArgumentParser(description=__doc__)
 parser.add_argument("value", nargs="?", help="Share URL, Markdown link, or raw shareRedId")
 args = parser.parse_args()
 try:
 value = args.value
 if value is None:
 value = input("Paste share URL or shareRedId: ") if sys.stdin.isatty() else sys.stdin.read()
 print(json.dumps(extract(value), ensure_ascii=False, indent=2))
 return 0
 except (ValueError, EOFError) as exc:
 print("Error: " + str(exc), file=sys.stderr)
 return 2


if __name__ == "__main__":
 sys.exit(main())

There is one offset per decoded byte. The 32-character token gives us 24 bytes, so zip processes all of them after the length check. Subtraction wraps modulo 256; the hexadecimal check then rejects values that don’t fit the expected encoding.

keep_blank_values=True keeps empty query parameters so I can reject them. Duplicate parameters also fail: there’s no good reason to guess which one was intended. The leading dot in the hostname suffix check prevents a lookalike domain from passing as a subdomain.

Re-encoding the recovered bytes should reproduce the token exactly. This checks the calculation, not the identity of the sender.

After saving the code above as rednote_share_id.py, run it with a raw token or a quoted long URL. With no argument it prompts for input; redirected standard input works too.

python3 rednote_share_id.py 'Mjc0Mzc6OkA-QEZETExFTDY0Ozo4OjxA'

Trying a sample token

Here is cnlnn’s artificial test vector, so you can try it without using a personal link:

Stage	Value
Input `shareRedId`	`Mjc0Mzc6OkA-QEZETExFTDY0Ozo4OjxA`
Base64URL-decoded bytes, shown as text	`27437::@>@FDLLEL64;:8:<@`
Recovered account ID	`0123456789abcdef01234567`
Re-encoded token	`Mjc0Mzc6OkA-QEZETExFTDY0Ozo4OjxA`

The odd-looking text in the second row is what comes straight out of Base64URL decoding. Subtract the offsets to get the ID; add them back and encode again to recover the input token.

Checking my own link

The output has two Boolean fields that are easy to confuse:

round_trip_matches: true means the decoded ID encodes back to the original token. It doesn’t tell us whether the account exists.
profile_verified: false is always false, because the script never loads the profile.

My own token passed the round-trip check. I opened the resulting URL in a browser and, sure enough, it was my profile. A plain HTTP fetch hadn’t returned the profile details, so I needed the browser for that check. The internal account ID was also different from the RedNote account number shown on my profile.

A forwarded link can still carry the original sharer’s token, though. If a mate sends me somebody else’s link, decoding it may lead to that person’s account rather than my mate’s.

A few regression checks

Save this beside rednote_share_id.py to check the expected result and a few inputs that should fail:

from rednote_share_id import extract

TOKEN = "Mjc0Mzc6OkA-QEZETExFTDY0Ozo4OjxA"
result = extract(TOKEN)
assert result["account_id"] == "0123456789abcdef01234567"
assert result["round_trip_matches"] is True
assert result["profile_verified"] is False

# Derive the real platform origin from the decoder's profile URL.
origin = result["profile_url"].split("/user/profile/")[0]
for value in (
 "", "abc", "A" * 32, TOKEN + "=",
 origin + "/explore/example",
 origin + "/?shareRedId=" + TOKEN + "&shareRedId=" + TOKEN,
):
 try:
 extract(value)
 except ValueError:
 pass
 else:
 raise AssertionError("Unexpectedly accepted: " + repr(value))

My full tests also cover copied Markdown, HTML-escaped query strings, short links and lookalike hostnames. The artificial token is enough to test the calculation, so I’ve kept my real share URL and account details out of the fixtures.

What the link gave away

I meant to share a post, but the URL also provided a route to my public profile. It didn’t expose a password or grant access to private posts. Still, that was more than I’d intended to send.

Removing the extra parameters is a separate job. Some may affect whether the post opens, so I’d want to check the cleaned link before relying on it. For this experiment, understanding how the account ID got there was enough.

Avoiding Random Ansible Runner Hangs in Async Multithreaded Python

Sun, 07 Jun 2026 00:00:00 +0000

I recently hit an annoying failure mode while running Ansible Runner from a Python program that uses both asyncio and worker threads. Most runs completed normally. Every now and then, a runner call just stopped making progress. No obvious exception. No clear reproduction. Just one of those “why is this process still here?” moments.

The clue was an old ptyprocess issue: . The maintainer note there describes the likely symptom as random-seeming deadlocks and hangs, which matched the behaviour almost too well.

Table of Contents

The shape of the problem

Ansible Runner uses pexpect around the child process it starts. That is convenient because pexpect can watch output, answer prompts, and keep the runner loop simple.

The uncomfortable part is the default PTY-backed spawn path. Under the hood, it relies on a fork()-to-exec() sequence with Python-level work in the middle. In a single-threaded program, this is usually boring enough. In a multithreaded program, it becomes much less boring.

After fork(), only the calling thread exists in the child process. Locks that were held by other threads may still look held, but the threads that could release them no longer exist. If the child then runs code that touches one of those locks before exec(), it can freeze in a place that looks unrelated to your application logic.

That is why this class of bug feels random. The failure depends on timing: which thread held what, exactly when the runner spawned a child, and what the child touched before exec().

Why `asyncio` made it easier to notice

asyncio itself was not the bug. The event loop was just the place where the symptom became visible.

In my case, runner work was launched from async workflows and delegated into threads so that blocking Ansible execution would not stop the event loop. That means the process had at least three moving parts:

an event loop scheduling higher-level tasks;
worker threads calling into Ansible Runner;
pexpect spawning and supervising child processes.

This is exactly the kind of runtime where a “mostly fine” fork-safety issue can turn into a real production headache.

The workaround I chose

The practical workaround was not to make the PTY path safer. I replaced it at the integration boundary.

pexpect also has , which uses subprocess.Popen instead of a PTY-backed spawn. For my use case, I did not need a real terminal. I needed a child process, stdout/stderr handling, pattern matching, timeouts, and predictable termination. A subprocess-backed spawn was a better fit.

The patch had four parts:

Create a small spawn adapter on top of PopenSpawn.
Start each runner child in its own process group and make termination signal the whole group.
Preserve the lifecycle behaviour Ansible Runner expects, including timeout handling, process status, and resource cleanup.
Monkey patch the runner’s pexpect.spawn reference early enough that runner instances use the adapter.

The adapter is not just a constructor wrapper. It also carries the close, timeout, liveness, process-status, pipe-cleanup, and process-group termination semantics needed by the runner loop. This exact patch is for Linux and Unix-like systems: it relies on POSIX process groups through os.setsid, os.killpg, and preexec_fn. Stripped of project-specific logging, the relevant code is:

import os
import signal
import time
from collections.abc import Awaitable
from typing import Any

from pexpect.exceptions import ExceptionPexpect, TIMEOUT
from pexpect.popen_spawn import PopenSpawn


class RunnerSpawn(PopenSpawn):
 """An Ansible runner spawn."""

 def __init__(self, cmd: str, args: list[str] | None = None, **kwargs: Any) -> None:
 """Initialise an Ansible runner spawn.

 Parameters
 ----------
 cmd : str
 A command.
 args : list[str] | None, default: `None`
 Command arguments (default: `None`). Do not provide these if they have already been included in ``cmd``.
 **kwargs : dict[str, typing.Any], optional
 Keyword arguments.

 Notes
 -----
 ``preexec_fn`` defaults to `os.setsid` so that the subprocess and all its descendants share a process group.
 This ensures `close()` and `kill()` can terminate the entire process tree, preventing orphaned child processes
 when ansible-runner spawns nested subprocesses.
 """
 cmd: list[str] = [cmd]
 if args:
 cmd.extend(args)

 kwargs.setdefault("preexec_fn", os.setsid)
 try:
 super().__init__(
 cmd=cmd,
 **{
 key: value
 for key, value in kwargs.items()
 if key
 in (
 "codec_errors",
 "cwd",
 "encoding",
 "env",
 "logfile",
 "maxread",
 "preexec_fn",
 "searchwindowsize",
 "timeout",
 )
 },
 )
 except Exception as exception:
 raise ExceptionPexpect(f"Failed to initialise an Ansible runner spawn with the cmd: {cmd}") from exception

 def close(self, force=True) -> None:
 """Close an Ansible runner spawn.

 Parameters
 ----------
 force : bool, default: `True`
 A flag denoting whether to force closing an Ansible runner spawn after failed graceful termination (`True`,
 default) or not (`False`).

 Notes
 -----
 `ptyprocess.PtyProcess.terminate()` is a reference.
 """
 if self.closed:
 return

 try:
 if not self.isalive():
 return

 self.kill(sig=signal.SIGHUP)
 time.sleep(0.1)
 if not self.isalive():
 return

 self.kill(sig=signal.SIGCONT)
 time.sleep(0.1)
 if not self.isalive():
 return

 self.kill(sig=signal.SIGINT)
 if not force:
 return

 time.sleep(0.1)
 if not self.isalive():
 return

 self.kill(sig=signal.SIGKILL)
 except OSError:
 pass
 finally:
 try:
 if not self.isalive():
 proc = getattr(self, "proc", None)
 if proc is not None:
 for pipe_name in ("stdin", "stdout"):
 pipe: Any = getattr(proc, pipe_name, None)
 if pipe is None or pipe.closed:
 continue

 try:
 pipe.close()
 except OSError:
 pass
 finally:
 self.closed = True

 def expect(self, *args, **kwargs) -> Awaitable[int] | int | None:
 try:
 return super().expect(*args, **kwargs)
 except TIMEOUT:
 # `ansible_runner.runner.Runner.run()` contains a fix to expect an EOF in at most 5 seconds. After monkey
 # patching `ansible_runner.runner.pexpect.spawn`, a timeout exception causes unexpected resource leaks.
 # Reference: https://github.com/ansible/ansible-runner/issues/1330
 return None

 def isalive(self) -> bool:
 """Check if an Ansible runner spawn's process is alive.

 Returns
 -------
 bool
 A flag denoting whether an Ansible runner spawn's process is alive (`True`) or not (`False`).
 """
 if not (alive := self.proc.poll() is None):
 if self.proc.returncode >= 0:
 self.exitstatus = self.proc.returncode
 self.signalstatus = None
 else:
 self.exitstatus = None
 self.signalstatus = -self.proc.returncode

 return alive

 def kill(self, sig) -> None:
 """Send a signal to the process group of an Ansible runner spawn.

 Parameters
 ----------
 sig : int
 A signal.
 """
 os.killpg(os.getpgid(self.proc.pid), sig)

Then I patch the runner spawn entry point before creating runner objects:

ansible_runner.runner.pexpect.spawn = RunnerSpawn

The process-group detail matters because Ansible Runner may spawn nested subprocesses. Killing only the direct child can leave descendants behind. By setting preexec_fn to os.setsid, the adapter creates a fresh process group for the runner tree; by overriding kill() to call os.killpg(os.getpgid(self.proc.pid), sig), the close sequence signals the whole group.

Process termination is only part of cleanup. PopenSpawn owns the subprocess’s stdin and stdout pipes and uses a reader thread to consume output. Once the process is no longer alive, close() closes both pipes, allowing those resources and the reader to finish cleanly. It skips cleanup immediately when the spawn is already closed, only closes pipes after the child has exited, tolerates individual pipe-close failures, and still marks the spawn as closed in every path.

The liveness check also translates subprocess return codes into the status fields expected by pexpect: a non-negative return code becomes exitstatus, while a negative return code represents termination by a signal and becomes signalstatus. Keeping those two cases separate prevents a signal number from being mistaken for an ordinary process exit code.

The timeout handling is also part of the adapter on purpose. Ansible Runner has had fixes around draining final output until EOF so that events are not lost near process shutdown; see for the general shape of that race. After replacing the spawn backend, letting a TIMEOUT escape from expect() could short-circuit cleanup and leak resources, so the adapter returns None.

That is the main lesson here: the monkey patch is small, but the compatibility surface is not only spawn(cmd). It includes liveness, exit and signal status, timeout behaviour, pipe cleanup, idempotent closing, and process-tree termination. The replacement should look boringly compatible to the runner loop.

Things I would check before copying this pattern

This workaround is not a universal “replace pexpect with subprocess” rule.

It is a good fit when:

your runner commands do not genuinely require PTY behaviour;
you are embedding runner calls inside a multithreaded Python process;
your runtime is Linux or another Unix-like system with POSIX process-group semantics;
your hangs line up with child-process spawning rather than Ansible task logic;
you can patch the spawn path before any runner objects are created.

It is a poor fit when:

the child process needs terminal-specific behaviour;
your automation depends on exact PTY semantics;
you need the same code path to work on Windows;
you cannot control import order;
you have not tested cancellation, timeout, and signal handling.

Takeaway

When a Python program mixes asyncio, threads, and child-process orchestration, fork() details stop being trivia. A hang may not come from the async task you are staring at. It may come from a child process being born in a runtime where another thread held the wrong lock at the wrong time.

For my case, moving Ansible Runner away from the PTY-backed pexpect spawn path made the runner lifecycle much more predictable. The patch is intentionally unglamorous: keep Runner’s expectations intact, use a subprocess-backed spawn where a PTY is unnecessary, terminate the whole process group, and release the subprocess resources once it exits.

That is often the best kind of production fix. Not clever. Just less surprising.

Case Studies |

Exporting Local WeChat Voice Messages on macOS

Looking for the recordings

Reading the databases

Matching messages to audio

Database keys and live files

Running the example

List before exporting

Converting SILK to MP3

Keys, cleanup and testing

Complete example

What a RedNote Share Link Can Reveal

The parameter to look for

Make the decoder fussy

The Python script

Trying a sample token

Checking my own link

A few regression checks

What the link gave away

Avoiding Random Ansible Runner Hangs in Async Multithreaded Python

The shape of the problem

Why asyncio made it easier to notice

The workaround I chose

Things I would check before copying this pattern

Takeaway

Why `asyncio` made it easier to notice