wasm32-wasi-vfs.c
Overview
This file implements a minimal and simplified Virtual File System (VFS) interface designed for use with SQLite on Unix-like operating systems. The implementation is tailored to embedded environments where complex VFS features such as file locking, dynamic extensions, temporary files, and file truncation are either unnecessary or unsupported.
A key feature of this VFS is the buffering of writes to journal files to optimize write performance on devices that do not cache writes efficiently in the OS. Instead of performing many small writes, the VFS coalesces writes into aligned blocks before flushing them to disk.
The file exposes a custom sqlite3_vfs named "demo" that can be registered with SQLite using the function sqlite3_demovfs().
Data Structures
DemoFile
Description: Represents an open file handle within this VFS. This struct extends
sqlite3_fileand adds fields for buffering journal writes.Fields:
sqlite3_file base: The base class for SQLite file handles. Must be the first member.int fd: The native file descriptor obtained from the OS for the opened file.char *aBuffer: Pointer to a malloc-allocated buffer used for write buffering (only for journal files).int nBuffer: Number of valid bytes currently in aBuffer.sqlite3_int64 iBufferOfst: The file offset corresponding to the start of the data in aBuffer.
Constants and Macros
SQLITE_DEMOVFS_BUFFERSZ (default 8192): Size of the write buffer used for journal files.
MAXPATHNAME(512): Maximum supported file path length.F_OK,R_OK,W_OK: Used for file access checking, defined if missing.
Functions and Methods
demoDirectWrite(DemoFile *p, const void *zBuf, int iAmt, sqlite_int64 iOfst)
Purpose: Write data directly to the file descriptor at a specified offset, bypassing any buffering.
Parameters:
p: Pointer toDemoFile.zBuf: Pointer to data buffer to write.iAmt: Number of bytes to write.iOfst: File offset to write at.
Returns:
SQLITE_OKon success, or an appropriate SQLite I/O error code.Usage: Called internally when there is no buffer or when flushing the buffer.
demoFlushBuffer(DemoFile *p)
Purpose: Flush the buffered data in DemoFile.aBuffer to the underlying file descriptor.
Parameters:
p: Pointer toDemoFile.
Returns:
SQLITE_OKif successful or an error code if the write fails.Details: No-op if the buffer is empty or if the file has no buffer (non-journal files).
demoClose(sqlite3_file *pFile)
Purpose: Close an open file handle.
Parameters:
pFile: SQLite's generic file handle pointer.
Returns:
SQLITE_OKon success or an error code from flushing or close failures.Behavior: Flushes any buffered data, frees allocated buffer memory, and closes the OS file descriptor.
demoRead(sqlite3_file *pFile, void *zBuf, int iAmt, sqlite_int64 iOfst)
Purpose: Read data from the file at a specific offset.
Parameters:
pFile: SQLite file handle.zBuf: Buffer to fill with read data.iAmt: Number of bytes to read.iOfst: Offset in the file to read from.
Returns:
SQLITE_OKif successful, or an I/O error code.Details: Flushes any buffered writes before reading to ensure consistency.
demoWrite(sqlite3_file *pFile, const void *zBuf, int iAmt, sqlite_int64 iOfst)
Purpose: Write data to a file, buffering writes if this is a journal file.
Parameters:
pFile: SQLite file handle.zBuf: Data buffer to write.iAmt: Number of bytes to write.iOfst: File offset to write to.
Returns:
SQLITE_OKon success or an error code.Algorithm:
If the file has a buffer (aBuffer), data is copied into this buffer.
If the buffer is full or data is not sequential, the buffer is flushed before copying.
Otherwise, write is performed directly via
demoDirectWrite.
demoTruncate(sqlite3_file *pFile, sqlite_int64 size)
Purpose: Truncate a file to a specified size.
Parameters:
pFile: SQLite file handle.size: Desired file size.
Returns: Always
SQLITE_OK.Implementation Note: This is a no-op; truncation is not supported in this VFS.
demoSync(sqlite3_file *pFile, int flags)
Purpose: Sync file contents to persistent storage.
Parameters:
pFile: SQLite file handle.flags: Sync flags (ignored).
Returns:
SQLITE_OKon success or SQLITE_IOERR_FSYNC on failure.Behavior: Flushes buffer then calls fsync() on the file descriptor.
demoFileSize(sqlite3_file *pFile, sqlite_int64 *pSize)
Purpose: Get the file size in bytes.
Parameters:
pFile: SQLite file handle.pSize: Output parameter for file size.
Returns:
SQLITE_OKon success; error codes from flush or fstat.Details: Flushes buffer first, then uses fstat() to obtain size.
Locking Functions (demoLock, demoUnlock, demoCheckReservedLock)
Purpose: Stub implementations for locking; no-ops.
Parameters:
demoLockanddemoUnlockaccept lock level.demoCheckReservedLocksets output to 0 (no lock).
Returns: Always
SQLITE_OK.Note: This VFS does not implement file locking; it assumes single connection per database.
demoFileControl(sqlite3_file *pFile, int op, void *pArg)
Purpose: File control operations.
Returns: Always
SQLITE_OK.Note: No file control operations are implemented.
demoSectorSize(sqlite3_file *pFile) and demoDeviceCharacteristics(sqlite3_file *pFile)
Purpose: Return sector size and device characteristics.
Returns: Always 0.
Note: These would allow SQLite optimizations but are not implemented here.
demoOpen(sqlite3_vfs *pVfs, const char *zName, sqlite3_file *pFile, int flags, int *pOutFlags)
Purpose: Open a file within the VFS.
Parameters:
pVfs: Pointer to the VFS object.zName: File path to open.pFile: Pointer toDemoFilestruct to populate.flags: Open flags (e.g.,SQLITE_OPEN_READWRITE).pOutFlags: Output flags (optional).
Returns:
SQLITE_OKon success or an error code.Behavior:
Allocates a write buffer if opening a main journal file.
Maps SQLite open flags to POSIX flags.
Opens the file descriptor.
Initializes
DemoFilestructure.Assigns method table with file operations.
demoDelete(sqlite3_vfs *pVfs, const char *zPath, int dirSync)
Purpose: Delete a file and optionally sync the directory.
Parameters:
pVfs: VFS pointer.zPath: Path of the file to delete.dirSync: If non-zero, fsync the containing directory.
Returns:
SQLITE_OKif file deleted or does not exist; error otherwise.Implementation Detail:
Uses
unlink().If
dirSyncis set, opens directory and calls fsync() to ensure directory entry removal is flushed.
demoAccess(sqlite3_vfs *pVfs, const char *zPath, int flags, int *pResOut)
Purpose: Check file existence and permissions.
Parameters:
zPath: Path to check.flags: One of SQLITE_ACCESS_EXISTS, SQLITE_ACCESS_READ, or SQLITE_ACCESS_READWRITE.pResOut: Output boolean (1 if access allowed).
Returns: Always
SQLITE_OK.Implementation: Uses POSIX
access()with appropriate flags.
demoFullPathname(sqlite3_vfs *pVfs, const char *zPath, int nPathOut, char *zPathOut)
Purpose: Convert a relative path to a full path.
Parameters:
zPath: Input path.nPathOut: Size of output buffer.zPathOut: Output buffer.
Returns:
SQLITE_OK.Implementation: Copies input path as-is; does not resolve relative paths to absolute paths.
Dynamic Loading Functions (demoDlOpen, demoDlError, demoDlSym, demoDlClose)
Purpose: Stub implementations for dynamic library loading.
Behavior: No support; return null or set error message.
Note: Loading of dynamic extensions is omitted.
demoRandomness(sqlite3_vfs *pVfs, int nByte, char *zByte)
Purpose: Fill buffer with pseudo-random data.
Returns: Always
SQLITE_OK.Note: No real randomness is generated.
demoSleep(sqlite3_vfs *pVfs, int nMicro)
Purpose: Sleep for at least
nMicromicroseconds.Returns: The requested sleep time.
Implementation: Uses
sleep()andusleep().
demoCurrentTime(sqlite3_vfs *pVfs, double *pTime)
Purpose: Get current UTC time as Julian day.
Returns:
SQLITE_OK.Details:
Uses
time()rounded to seconds.Converts Unix epoch to Julian day.
Note on potential year 2038 problem due to
time_tsize.
sqlite3_demovfs(void)
Purpose: Return a pointer to the static
sqlite3_vfsstructure implemented here.Usage: Register this VFS with SQLite using
sqlite3_vfs_register(sqlite3_demovfs(), 0);.VFS Name:
"demo"Version: 1
File Size:
sizeof(DemoFile)Methods: Implements all required file methods (open, delete, access, etc.).
Test and Initialization Functions (Conditional on SQLITE_TEST)
TCL commands register_demovfs and unregister_demovfs allow dynamic registration/unregistration of the VFS in TCL test environments.
sqlite3_os_init() registers the demo VFS at startup.
sqlite3_os_end() is a stub.
Implementation Details and Algorithms
Buffered Journal Writes: The VFS buffers writes to journal files in an 8KB (default) buffer aligned with filesystem sectors. This reduces the number of small write operations, which is critical for performance on embedded devices without OS write caching.
Write Buffer Management: Writes are accumulated in the buffer; if the buffer is full or a non-sequential write occurs, the buffer is flushed to disk.
No Locking: All locking calls are no-ops, delegating responsibility for concurrency control to the user.
Limited Feature Set: The VFS omits features like dynamic extensions, file truncation, and temporary file handling to maintain simplicity.
File Deletion Sync: When deleting files, the containing directory is optionally synced to ensure directory entry removal is persisted.
Interaction with Other System Components
Uses POSIX system calls extensively:
open(), read(),write(),close(),lseek(), fsync(),unlink(), fstat(), andaccess().Interacts with SQLite core by implementing the
sqlite3_vfsinterface and providing file I/O methods.When used, this VFS replaces or supplements SQLite’s default VFS for testing or embedded environment scenarios.
Visual Diagram: Function Flow and File Operation Relationships
flowchart TD
A[sqlite3_demovfs] -->|Register VFS| B[demoOpen]
B --> C[Allocate buffer if journal]
B --> D["open() system call"]
B --> E[Assign method pointers]
E --> F[demoRead]
E --> G[demoWrite]
E --> H[demoClose]
E --> I[demoSync]
E --> J[demoFileSize]
E --> K[demoDelete]
E --> L[demoAccess]
E --> M[demoFullPathname]
G --> N{Has buffer?}
N -->|Yes| O[Buffer writes]
N -->|No| P["Direct write (demoDirectWrite)"]
O --> Q{Buffer full or non-sequential?}
Q -->|Yes| R["Flush buffer (demoFlushBuffer)"]
Q -->|No| S[Copy data to buffer]
H --> T[Flush buffer]
H --> U[free buffer]
H --> V[close file descriptor]
K --> W["unlink()"]
W --> X{dirSync?}
X -->|Yes| Y[fsync directory]
X -->|No| Z[Return]
subgraph File Operations
F
G
H
I
J
K
L
M
end
Usage Examples
Registering the VFS with SQLite
sqlite3_vfs *pVfs = sqlite3_demovfs();
sqlite3_vfs_register(pVfs, 0);
After registration, SQLite can open and use files through this VFS by specifying "demo" as the VFS name.
Opening a Journal File with Buffered Writes
sqlite3_file file;
int flags = SQLITE_OPEN_MAIN_JOURNAL | SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE;
int rc = demoOpen(pVfs, "journalfile", &file, flags, NULL);
if (rc == SQLITE_OK) {
// Writes to this file will be buffered for performance
}
Writing Data with the Buffering Logic
When SQLite writes to the journal, the VFS buffers the data in memory and flushes it only when the buffer is full or on sync, improving efficiency on embedded systems without OS write caching.
References to Related Topics
The general behavior of SQLite VFS implementations is detailed in SQLite VFS Interface.
Buffer management and write coalescing techniques are discussed in File I/O Optimization.
Handling of rollback journals in SQLite is covered in SQLite Journal Files.
Usage of POSIX system calls for file operations can be found in POSIX File APIs.