Sockets & networking

This reference describes the implemented native runtime. Historical paths and internal version labels are retained where relevant. Later extension chapters supersede older limitations.

Linux sockets are implemented in restored/sockets.inc, with isolated DNS/TLS helpers. This is a restored-runtime extension, not recovered historical syntax. This document describes the original synchronous bounded TCP/DNS/TLS client APIs. Stage 1 additionally implements multi-handle readiness, nonblocking progress operations, staged TLS listeners and UDP in socket_foundations.inc/socket_udp.inc: see ../NETWORK-FOUNDATIONS.md for those contracts and current evidence. No plugin ABI is supplied. Existing HTTP/S remains the separate curl-backed http family; socket CLI TLS options do not alter HTTP/S.

Permissions and addresses

Every socket operation requires --allow-network. Creating a listener additionally requires --allow-listen; that flag alone grants no networking. These flags are feature gates, not a sandbox. connect is plaintext; tlsconnect is verified TLS. Non-loopback numeric/hostname destinations are allowed with the network flag. Explicit non-loopback/wildcard binds are allowed with both flags and can expose a service; the shipped examples and integration fixtures bind loopback only. No service is deployed or left running.

Hosts can be strict numeric IPv4/IPv6, exactly "localhost" (mapped to "127.0.0.1", without DNS or IPv6 fallback), or validated DNS names for outbound connects. Listen addresses remain numeric/localhost only. DNS names use ASCII letters/digits/hyphens, 1–63 byte labels and at most 253 bytes overall, no empty labels or initial/final label hyphens, and at least one letter. NULs, bracketed IPv6, scope/zone suffixes, trailing dots, underscores, Unicode and invalid all-numeric IP-looking names are rejected before creating a descriptor. Supply already encoded ASCII IDNA names if needed; no IDNA conversion is performed. No blocking getaddrinfo or resolver threads are used. DNS and sequential connection attempts share the connect deadline; see the detailed DNS/TLS section below. IPv6 literals such as "::1" work if the host kernel supports IPv6; failure is an ordinary explicit socket error, never a silent fallback. IPv6 listeners set IPV6_V6ONLY and do not also listen on IPv4.

Syntax

Declare every destination with var first. Destinations must currently be scalar variables, not native arrays or JSON values. Multiple destinations must differ. All destinations, scalar inputs, integers and handle kinds are validated before network effects. The command's arity and destination syntax are validated when loading the whole script.

socket connect $connection "127.0.0.1" 12345 1000
socket tlsconnect $secure "good.test" 12345 1000
socket listen $listener "127.0.0.1" 0 4
socket accept $connection $listener 1000
socket sendall $connection "hello\n" 1000
socket recv $data $eof $connection 4096 1000
socket readline $data $eof $connection 4096 1000
socket endpoint $address $port $connection
socket close $connection

Handles are runtime-owned opaque strings, not OS descriptor numbers. Wrong-kind, unknown and closed handles are rejected. Never infer their representation or treat them as security tokens. Copying a scalar handle aliases the same connection; it does not duplicate the descriptor. Overwriting the last scalar copy does not close it. Close explicitly or let runtime teardown close everything. Function returns and catch blocks do not automatically close live handles; ordinary errors generally leave them usable. Every script test runs with a fresh runtime, independent handles and budgets, and destroys its sockets after either pass or failure.

EOF, lines, buffering and failures

• $eof is "1" only when EOF has actually been observed and the returned bytes exhaust the shared input buffer; otherwise it is "0". Receiving the peer's last nonempty chunk usually still returns "0", followed by an empty "1" on a subsequent call. There is no speculative read solely to discover EOF.
• An empty terminated line returns "", "0". Empty raw data with "1" means EOF. Positive maxbytes prevents ambiguous zero-length recv requests.
• At EOF, a partial unterminated line is returned with $eof = "1". Repeated reads at EOF return empty data with "1" (subject to budgets).
• recv and readline share one buffer. A line read may prefetch several coalesced lines; mixing the operations preserves exact byte order. Raw reads make no packet/message-boundary guarantees.
• A plaintext TCP readline timeout preserves accumulated bytes for a later call, and recv can drain them immediately. TLS transport timeouts/errors instead invalidate the handle and discard its buffer. Output variables remain unchanged on either transport's read error.
• Reaching maxbytes without LF is a catchable socket line limit error. The entire accumulated buffer is preserved, no line prefix is returned or discarded, and no extra byte is read to test for EOF. Thus an exactly-limit unterminated line also errors, even if the peer has closed. Drain with recv, retry with a larger limit up to 65536, or close the handle. A previously prefetched LF beyond the new limit also errors without consuming data.
• Timeouts, refused connections, reset/disconnected peers, unsupported addresses, bind conflicts, invalid types/arguments and stale handles are catchable script errors. There is no SIGPIPE process termination: sends use MSG_NOSIGNAL.
• A send failure after the send loop starts includes sent=N of M bytes (do not blindly retry). N counts plaintext bytes confirmed by this operation's successful local send or SSL_write calls, not delivered bytes (a failed TLS write may have emitted additional encrypted bytes), and may be zero or partial. Previously completed sends are not included. Retrying the entire payload can duplicate data; recovery/framing is the application protocol's responsibility. Preflight validation/resource failures happen before any send and do not claim partial progress.
• Resource-budget failures are fatal and cannot be swallowed by try/catch. OS memory/descriptor exhaustion is also fatal. Owned descriptors are released before emitting OS-exhaustion diagnostics; other fatal errors close resources during runtime unwinding.

Deadlines and limits

All descriptors are RAII-owned, nonblocking and close-on-exec. poll waits use one monotonic absolute deadline for the entire operation, including all short reads/writes and EINTR/EAGAIN retries; progress does not reset the deadline. A deadline is not a hard real-time scheduling guarantee: descheduling and bounded local bookkeeping can delay return. There is no unbounded blocking socket syscall, sleep-based network polling or resolver thread.

Before each OS read, room for its entire requested chunk must remain in the byte budget. Consequently a small final read or EOF may be refused when the remaining budget is smaller than the requested chunk. Read buffers never exceed 65536 logical bytes per handle. The general source, steps, string, frame and output limits remain in force. Kernel TCP buffers are kernel-managed, not included in the interpreter's string budget; the live-handle bound also bounds the number of such buffers. Budgets cannot be reset by closing sockets, catching errors or overwriting variables. HTTP has its existing separate budgets.

Runnable loopback-only examples

examples/echo_server.v binds an ephemeral IPv4 loopback port, exclusively creates the readiness file named by its sole argument, accepts one client, echoes one LF-terminated line and exits. examples/echo_client.v takes the numeric port as its sole argument, sends "hello Vanilla\n", verifies the exact reply and then verifies EOF.

Run the verified wrapper (it starts both compiled interpreter processes, discovers the ephemeral port from a new file, bounds all waits, checks stdout/stderr and file contents, and always reaps the server):

cd "/root/vanilla1"
make release
python3 "tests/run_sockets_demo.py" "./build/vanilla1"

Exact application output checked by the wrapper:

# Client stdout:
echo: hello Vanilla
# Server stdout:
server received: hello Vanilla

For manual use in separate terminals, choose a readiness filename that does not already exist:

"./build/vanilla1" --allow-network --allow-listen --allow-files run "examples/echo_server.v" -- "build/new socket port.txt"

Read the decimal port from that file and, within the server's five-second accept deadline, pass it to:

"./build/vanilla1" --allow-network run "examples/echo_client.v" -- "PORT_FROM_FILE"

The placeholder must be replaced by that actual decimal port. The wrapper is the convenient reproducible route and requires no fixed port or public listener.

DNS resolution policy and bounds

--dns-server 127.0.0.1:5353 or --dns-server '[::1]:5353' explicitly selects one numeric resolver and port for socket hostname lookups. No hostnames, comma-separated lists, zero ports or implicit test overrides are accepted. This option requires --allow-network even if the script makes no calls. Without it, c-ares reads the system resolver configuration; only its first configured server is used. This deliberately trades resolver failover for a small predictable query scope. localhost and numeric IPs never query DNS.

A new c-ares channel per hostname operation performs one ares_getaddrinfo(AF_UNSPEC) lookup, normally A and AAAA. DNS only: no hosts-file/alias/search-domain expansion. ARES_AI_NOSORT avoids c-ares' address-sort probe connections. The runtime copies up to 16 candidates; an oversized result is a fatal error, not a silent truncation. It tries candidates sequentially in returned order until a TCP connection succeeds. Ordinary refusal/unreachable errors advance to the next candidate; a slow first candidate can consume the remaining deadline. There is no Happy Eyeballs/concurrent connection racing, re-resolution, or retry after TLS verification failure.

Additional conservative limits: 128 hostname lookup attempts per runtime, 8 tracked resolver descriptors, 16 resolver sendv attempts per lookup (including retries/partial sends), 1024 poll/process turns per lookup. Resolver tries=1, no search, and first-server-only keep query work finite; c-ares may internally use TCP for a truncated DNS reply. These are separate from the 32 application-handle limit and 16 MiB application-byte budget. Resolver networking uses nonblocking/CLOEXEC sockets and MSG_NOSIGNAL. No event thread is enabled. Poll waits respect both ares_timeout and the one caller deadline. Callback state and fixed candidate/socket arrays remain alive through ares_cancel and ares_destroy; callbacks neither throw nor start new queries.

DNS is unauthenticated and can redirect plaintext TCP; TLS verifies the original requested identity, never a DNS CNAME. Bounds on copied candidates, I/O attempts and event-loop turns are not bounds on all c-ares internal parsing/allocation work. A single library processing call, trust/configuration file reads, cryptography and OS scheduling are cooperative parts of the deadline, not preemptible hard-real-time work. Local configuration/CA paths can reside on slow or hostile filesystems; the runtime does not claim to time-limit those file syscalls. Use external process/time/memory limits for untrusted workloads.

TLS client trust and transport semantics

The operator may supply --tls-ca-file /path/to/private-ca.pem together with --allow-network. It replaces the default CA sources for socket TLS only, does not require the script file-write gate, and does not modify system trust. The path is explicit operator-authorized read access, not a script-supplied trust bypass. Without it, OpenSSL default trust paths are used, including its inherited operator-controlled SSL_CERT_FILE/SSL_CERT_DIR configuration. These options do not alter curl-backed HTTP/S. An absent/unreadable/invalid explicit bundle fails; there is no fallback to default trust after such a failure.

• Mandatory SSL_VERIFY_PEER chain verification plus the original DNS name (SSL_set1_host) or IP SAN (X509_VERIFY_PARAM_set1_ip_asc). DNS SNI only, never IP SNI. localhost maps to IPv4 for transport but still verifies localhost and sends that DNS SNI.
• TLS 1.2 minimum, OpenSSL default cipher/security policy, no compression, no renegotiation, no application session cache; certificate-list input capped at 128 KiB and verification depth at 8 intermediate certificates. No client certificate, early application data, insecure flag, pinning, custom verify callback, OCSP/revocation fetching or plaintext fallback is supplied.
• Handshake starts only after TCP connects, using the same absolute deadline as DNS and all candidate attempts. A failed handshake never stores a new handle or sends script application data.
• sendall, recv and readline dispatch through the transport attached to the handle. TLS SSL_read is attempted before polling, so buffered decrypted data (SSL_pending) is not mistaken for an idle socket. WANT_READ/WANT_WRITE wait on the requested direction under the unchanged operation deadline; each retry uses identical buffers/lengths. Partial-write mode advances only on positive completed writes. Empty sends do not call SSL_write.
• A custom socket BIO suppresses SIGPIPE with MSG_NOSIGNAL for handshake, application traffic and shutdown; no process-wide signal disposition is changed.
• Any TLS transport timeout/error makes the handle stale and closes the connection, discarding its buffer. All scalar aliases become stale; outputs remain unchanged on read failure. This avoids returning OpenSSL to script code with unfinished retry invariants. Preflight errors (wrong destinations/types/sizes) and line-length errors preserve the connection. TCP timeout buffering semantics remain unchanged.
• close_notify produces ordinary EOF with the existing buffered/partial-line semantics. Abrupt TCP EOF/reset is a TLS error, not clean EOF; no SSL_OP_IGNORE_UNEXPECTED_EOF. Successfully returned earlier data is not revoked, so applications still need framing/completeness checks.
• Explicit close attempts SSL_shutdown once without blocking, then frees TLS state and closes the FD. It does not wait for the peer's reciprocal shutdown or guarantee delivery of close_notify under backpressure. Error/unwind teardown does not attempt TLS I/O.
• TLS failures use fixed sanitized messages, not peer certificate text, payloads, OpenSSL error-stack dumps or secret paths. OS/library memory and descriptor exhaustion are fatal. Resource paths release owned FDs before constructing error objects so sanitizer diagnostics can function.

Existing string/frame/input-buffer budgets count plaintext application data, not TLS ciphertext, handshake bytes, CA certificates or c-ares/OpenSSL internal allocations. Each TLS connection owns its own context and SSL object. Kernel buffers and library storage are outside the 1 MiB interpreter input-buffer reservation. The 32 shared application-handle cap bounds their count, not a promised whole-process RSS.

Finite local IRC demo

python3 tests/run_irc_demo.py ./build/vanilla1
python3 tests/run_irc_demo.py ./build/vanilla1-asan

The wrapper generates a disposable CA/certs with installed OpenSSL, starts loopback-only DNS and TLS IRC fixtures on ephemeral ports, and launches the actual examples/irc_client.v with explicit good.test/port arguments and CA/DNS CLI options. It checks exact NICK/USER registration and three PONG responses, including trailing tokens with spaces, a source prefix, IRCv3 tags and lowercase command. The script displays messages only; it never executes server-provided commands, joins channels, sends chat, accepts credentials, or reconnects. It has no default endpoint. It reads at most 16 protocol lines of at most 512 bytes, with finite per-operation timeouts and total reservations below the runtime cap. CRLF framing is checked and embedded CR/NUL is rejected before a PONG can be formed. Server text is printed as untrusted text, not interpreted code; use a suitable output viewer for untrusted terminal escape sequences.

Verified output:

:fixture 001 vanilla_demo :Welcome locally
PING :token with spaces
:fixture PING fixture :second token
@time=local :fixture ping :tagged token
:fixture NOTICE vanilla_demo :No remote execution: socket close $h

Registration is NICK vanilla_demo\r\nUSER vanilla_demo 0 * :Vanilla read-only demo\r\n. Replies are exactly PONG :token with spaces\r\n, PONG fixture :second token\r\n, and PONG :tagged token\r\n. The fixture then sends close_notify; the script observes EOF and closes, and the fixture verifies reciprocal TLS shutdown followed by TCP EOF. All processes and fixture threads are bounded and reaped/joined. There was no public IRC interaction or remote deployment.

DNS/TLS verification (previous milestone)

python3 tests/verify_dns_tls.py
# Includes fresh warning-clean builds and all prior/new suites on both artifacts.
# Standalone new suite (optional final argument is a case-name substring):
python3 tests/test_dns_tls.py ./build/vanilla1
ASAN_OPTIONS=detect_leaks=1:halt_on_error=1 UBSAN_OPTIONS=halt_on_error=1 python3 tests/test_dns_tls.py ./build/vanilla1-asan
python3 benchmarks/dns_tls_smoke.py
# Normal targets also include the new tests and IRC wrapper:
make test test-sanitize

Final verifier: 2343/2343 per binary on release and ASan/UBSan, with no compiler warnings or sanitizer diagnostics in passing runs. Counts per artifact: core 183; cache differential 339 against each of pre-optimization, pre-utilities, pre-worklog, pre-sockets and pre-dns-tls; utilities 173; permissions 16; worklog 130; TCP 73; DNS/TLS 70; utilities/echo/IRC demos one each. The five cache runs repeat the same dataset against different baselines; this is not 2343 distinct programs. An additional pre-dns-tls baseline TCP suite passed 73/73, outside those per-candidate totals. All 15 original/baseline files retain their captured SHA-256 hashes.

New cases use real local DNS replies and actual certificate verification: A/AAAA, CNAME, first-address refusal then successful next address, NXDOMAIN, delayed/dropped/malformed DNS, shared DNS/handshake deadline, cancellation and repeated descriptor cleanup, candidate/query limits and real descriptor exhaustion; DNS/IP/SNI, TLS 1.2/1.3, expired/untrusted/wrong-name/IP certificates, handshake stalls, plaintext endpoint rejection, fragmented/coalesced/NUL traffic, SSL-buffered data, clean close_notify versus abrupt EOF, repeated stale-handle errors, validation before effects, 1 MiB exact transfer, backpressure with a nonzero partial-send count, resets/SIGPIPE, shared resource limits and real CA-file descriptor exhaustion. The prior 73-case suite retains all six invalid-host checks: two formerly valid-but-unsupported names were replaced by malformed labels, while accepted names moved to deterministic local DNS tests instead of public queries.

One initial ASan run exposed descriptor-exhaustion diagnostics being constructed before existing sockets were released in the resolver helper. The minimal failing case was retained; resource paths now release FDs before constructing Error. The untouched failure log is build/dns-tls-initial-descriptor-failure.log. No sanitizer was disabled, filtered or hidden. Certificate fixture setup also initially reused a CA subject rejected by OpenSSL's issuance database; distinct fixture subjects fixed setup, without weakening verification.

Evidence: dns-tls-verification.json, dns-tls-tests*.json, irc-demo*.json, dns-tls-preservation.json, and build/dns-tls-*.log. Fixed408738 exact outputs matched pre-dns-tls on all runs: 0.60012431523443521, 1.26496032951999. One warmup plus three alternating pinned runs: baseline median 0.230475375 s, candidate 0.233705827 s. Raw measurements/hashes: dns-tls-smoke.json. This is a short shared-host smoke, not a tuning campaign or performance guarantee.

Limitations of this evidence: no public endpoints, production deployment, large fuzzing campaign, hostile DNS flood benchmark, DNSSEC, DNS TCP-truncation fallback fixture, deterministic TLS key-update-induced opposite-direction WANT test, or forced allocator failure. WANT directions are both handled in code; real stalls/backpressure exercise nonblocking retry paths but do not prove every OpenSSL protocol-state transition. Dependency shared libraries are distribution binaries rather than sanitizer-instrumented builds. Independent parent specification/quality review remains appropriate.

Dependencies inspected: c-ares 1.27.0 (libc-ares-dev, pkg-config libcares) and OpenSSL 3.0.13 (libssl-dev, pkg-config openssl), GCC 13.3.0. The local package install also updated libcares2 to the distribution's 1.27.0 security revision; no remote machine or trust store was modified. API references used: installed /usr/include/ares.h and matching packaged manpages for ares_getaddrinfo, ares_init_options, ares_timeout, ares_cancel, socket callbacks and server configuration; official https://c-ares.org/docs/ares_getaddrinfo.html, ares_timeout.html, ares_cancel.html; versioned https://docs.openssl.org/3.0/man3/SSL_set1_host/, SSL_CTX_set_verify/, SSL_connect/, SSL_read/, SSL_write/. Current c-ares online pages include newer reentrancy guarantees; the implementation does not depend on those newer guarantees or use deprecated ares_getsock polling.

Previous TCP milestone verification

Historical evidence below predates the pre-dns-tls snapshot. Use verify_dns_tls.py above for current reproduction; the older preservation manifest intentionally covers only the older baseline set.

cd "/root/vanilla1"
python3 "tests/verify_sockets.py"
python3 "benchmarks/sockets_smoke.py"
# Ordinary Makefile targets include the new TCP tests and two-process demo:
make test test-sanitize

The verifier builds release and ASan/UBSan binaries and runs every prior release suite, prior cache differentials, TCP tests, both demos and additional cache differentials against baselines/pre-sockets/vanilla1. Each subprocess has a timeout. Python peers use loopback, finite socket timeouts, teardown events and joined threads. IPv6 tests are required, not silently skipped; they passed on this host. The permission-denial suite requires the existing root-to-nobody fixture prerequisites.

Verified counts per binary: core 183; cache differential 339 against each of pre-optimization, pre-utilities, pre-worklog and pre-sockets; utilities 173; OS permissions 16; worklog 130; TCP 73; utilities demo 1; two-process TCP demo 1. That is 1933 passing checks per binary, including repeated differential datasets against different baselines, not 1933 unique cases. All passed with leak detection and sanitizer halt-on-error enabled. Build log contains no compiler warnings.

The 73 TCP cases include fragmentation/coalescing, mixed reads, binary NULs, EOF and partial lines, backpressure with a verified nonzero partial-send count, 1 MiB exact transfer, resets/refusals, permissions, destination validation before connect/accept/read effects, invalid inputs, wrong/stale handles, handle/call/time/byte/buffer limits, real OS descriptor exhaustion, caught failures, runtime EOF and failed-test listener cleanup.

Evidence: docs/sockets-verification.json, docs/sockets-tests.json, docs/sockets-tests-asan.json, docs/sockets-demo*.json, docs/sockets-preservation.json, and the named build/sockets-*.log files. All 13 files under original/ and baselines/ match their before-task SHA-256 hashes. The baseline source archives and binaries were never modified.

Fixed408738 math smoke: exact values 0.60012431523443521 and 1.26496032951999 matched the preserved pre-sockets binary on every run. One warmup plus three alternating measured runs per binary, pinned to one available CPU: median pre-sockets 0.241372674 s, sockets 0.231556758 s. These short shared-host observations are not a performance campaign or a speedup claim; raw times, hashes and CPU are in docs/sockets-smoke.json.