This roadmap outlines the planned development for the NURL (Neural Unified Representation Language) project. Tasks are categorized and prioritized to evolve NURL into a robust, high-performance, and LLM-friendly systems language.
NURL has achieved a stable self-hosted compiler (Grammar v1.7) targeting LLVM. It features single-owner memory management with auto-drop, a functional standard library (JSON, HTTP client/server, Crypto, Threads/Mutex/Cond, Channels, Regex, MCP, Anthropic Claude API), and supports native (Linux/Win/macOS) and WebAssembly targets. HTTP server stack: Phases 1–6 + 5.3 (thread pool) + 5.4 (keep-alive, ~38× speedup on hot endpoints) + 7 (mime/static/auth/cookies/form) + 8 mostly (access log, Prometheus metrics, idle timeout, graceful-shutdown via SIGINT/SIGTERM/Ctrl+C) + 9 partial (multipart/form-data file uploads + reverse-proxy / streaming pass-through for AI gateways) shipped; Phase 8 leftovers (per-request total timeout, configurable parser limits) and Phase 9 leftovers (TLS, HTTP/2, WebSocket) remain.
Enhancing the type system and compiler efficiency.
pub keyword and implicit private scope for module-level symbols.i8, i16, i32, u16, u32, u64, and f32 (requires multi-character tokenization in the lexer).musttail support for deep recursion optimization.printf).zext, sext, and trunc for all sized types.? T Some payload for multi-field structs (mirrors the ! T E Ok-arm heap-box solution).Channel[T], Vec[Thread]).Vec[i] backing — see Metrics in http_middleware.nu). Either document the rule clearly or move to pass-by-handle for ~-bound multi-field structs.: ~ NetErr last_err miscompiles (i64 stored without insertvalue wrapper). Either fix codegen or document the workaround (sentinel-flag pattern).compiler/nurlc.py is retired once self-host is stable, or kept as a verification reference.runtime.c into bootstrap helpers (compiler-internal) vs. stdlib-FFI (numerics, libm, libcurl, sockets) for cleaner ownership.compiler/nurlc_lastgood.nu automation: Document when this snapshot is updated and integrate into the build pipeline.Rounding out the standard library for production use.
stdlib/ext/sqlite.nu) and PostgreSQL support.dir_create_all, dir_remove_all), file handles for chunked reading (file_read_chunk h n, file_readline h).Serialize[T] and Deserialize[T] traits for JSON, MsgPack, and TOML.stdlib/ext/csv.nu.Slice[A]): Implement a bounds-safe view type for vectors to replace raw pointer access.Time { year, month, day, hour, min, sec, ns, wday } struct + time_from_unix / time_to_unix (Howard Hinnant's civil_from_days / days_from_civil algorithms — proleptic Gregorian, branch-free, full i64 range), time_now, time_format_iso (2026-05-06T17:30:45Z), time_format_http (RFC 7231 IMF-fixdate Wed, 06 May 2026 17:30:45 GMT), time_parse_iso → ! Time ParseErr (offsets +HH:MM/-HH:MM/Z/z, fractional seconds ignored). UTC only — no timezone/DST handling.stdlib/std/arena.nu bump allocator for parser/AST/regex-NFA workloads (arena_new, arena_alloc, arena_reset, arena_free).sha512, blake3, md5, hmac_sha512 via libsodium-FFI or self-contained impl.bytes_read_u16_be/le, bytes_read_u32_be/le, bytes_read_u64_be/le (depends on sized unsigned types).log_info_kv for key-value pairs, JSON output mode for log lines.Path { String inner } type with path_canonical (via realpath) and path_relative_to base.serve_static s dir HttpRequest → HttpResponse — .. rejection, root → index.html, refuses directories via file_size IoErr; symlink containment deferred (operators pre-resolve dir).mime_for_ext s ext → s — 30+ extensions (text/image/font/audio/video/document/archive), case-insensitive, application/octet-stream default.parse_basic_auth → ? BasicAuth, parse_bearer_auth → ? String (RFC 7617, splits on first : so passwords with : work).request_cookie → ? String (RFC 6265 §5.4), response_set_cookie with CookieOpts { path, domain, max_age, secure, http_only, SameSite }.parse_form_urlencoded ( Vec u ) → ( Vec QueryPair ) in stdlib/ext/http_request.nu — byte-buffer scanner so embedded NUL bytes pass through, percent-decoding + + → space substitution, bare keys yield empty value, only the first = per segment splits, trailing & does not produce phantom pairs. Convenience: request_form_pairs HttpRequest → ? ( Vec QueryPair ) checks Content-Type: application/x-www-form-urlencoded (case-insensitive, tolerates ; charset=…). Multipart deferred to Phase 9.tcp_set_timeout after accept (default 30 s, server_new_with_timeout to override) — Slowloris defence comes for free.with_access_log) — [req] METHOD path → STATUS BYTESB MSms to stderr.Metrics + with_metrics + metrics_handler + metrics_render text-exposition format).nurl_signal_install_shutdown runtime fn (POSIX sigaction SIGINT/SIGTERM + Win32 SetConsoleCtrlHandler for CTRL_C/BREAK/CLOSE/LOGOFF/SHUTDOWN) → calls shutdown(fd, RDWR) on stored listener handle. NURL: signal_install_shutdown TcpListener → v + signal_clear_shutdown + signal_trigger_shutdown (test helper) in stdlib/std/signal.nu.http_request.nu).http_full.nu aggregator — shipped 2026-05-07: stdlib/ext/http_full.nu is a single-line entry point that $-includes every HTTP-stack module shipped through Phases 1–9: std/net, std/signal, ext/http, ext/http_json, ext/http_request, ext/http_response, ext/http_server, ext/http_router, ext/http_static, ext/http_auth, ext/http_middleware, ext/http_multipart. Cost model: zero impact on programs that don't include it (NURL's $ is opt-in per source file); programs that DO include it pay ~14 KB / ~3.6% extra binary size vs explicit per-module imports because NURL emits @-functions with external linkage so LLVM's link-time DCE cannot strip across the boundary aggressively. The lean per-module includes remain available — pick http_full.nu for prototypes and demos, pick explicit per-module includes when binary size matters (embedded targets, network-served WASM). examples/static_server.nu simplified from 10 explicit includes to a single $ \stdlib/ext/http_full.nu`(+std/fs.nu` for the boot-time index.html setup) — unchanged end-to-end behaviour.stdlib/ext/http_multipart.nu (~370 LOC pure NURL). MultipartPart { String name, String filename, String content_type, ( Vec u ) data }. Surface: parse_multipart_form ( Vec u ) body s boundary → ( Vec MultipartPart ) (low-level, byte-buffer in / OWNED Vec out), request_multipart_parts HttpRequest req → ? ( Vec MultipartPart ) (Some when Content-Type is multipart/form-data with boundary parameter, else None — handles quoted boundary, mixed-case media type, optional charset parameter), multipart_part_free / multipart_parts_free / multipart_count / multipart_find_first ( Vec MultipartPart ) parts s name → i (-1 if absent). RFC 7578 + 2046 conformance: preamble before opening boundary correctly ignored; truncated body returns parts that completed before the cut (best-effort posture mirrors parse_form_urlencoded); part without Content-Disposition: form-data; name=... silently dropped; missing per-part Content-Type defaults to text/plain; binary part data preserved including NUL bytes (file-upload pathway). 15-case offline test compiler/tests/http_multipart.nu (two text fields / file upload with binary data / preamble / truncated body / missing-disposition drop / default content-type / empty boundary / no-marker body / 6 request_multipart_parts cases including boundary="quoted" + Multipart/Form-Data capitalisation + ; charset=utf-8 extra parameter / multipart_find_first lookup). Carve-out added to run_tests.bat + run_tests.sh. Pure NURL — no runtime/compiler changes. Strategic milestone: completes the static-file lifecycle — serve_static (download, Phase 7) + multipart (upload, Phase 9) gives NURL HTTP servers full HTML-form-with-files support out of the box.stdlib/ext/http_proxy.nu (~330 LOC) wires libcurl multi streaming (runtime §14b) through to the HTTP-server chunked writer (stdlib/ext/http_response.nu) so a NURL program can sit in front of any upstream (Anthropic / OpenAI / Google / Ollama) and pump SSE / token streams back to the client in real time. Surface: proxy_default_opts → ProxyOpts { i timeout_ms (60s), i connect_timeout_ms (10s), b strip_hop_by_hop, b preserve_host_header, b strip_content_encoding, b strip_content_length }; : | ProxyErr { ProxyUpstream ProxyClientWrite ProxyOther } + proxy_err_name; streaming core proxy_stream_to_conn[_with] TcpConn conn HttpRequest req s upstream_url [ProxyOpts opts] → ! v ProxyErr (opens upstream stream → pumps headers → response_begin_chunked → forwards each upstream chunk via response_write_chunk → response_end_chunked); dedicated server proxy_serve_run[_with] TcpListener listener s upstream_base [opts] → ! v NetErr (accept loop calling streaming proxy on every request). Header policy: RFC 7230 §6.1 hop-by-hop set (Connection, Keep-Alive, Proxy-Authenticate, Proxy-Authorization, TE, Trailer, Transfer-Encoding, Upgrade) stripped both directions; Host dropped from forward (libcurl rebuilds from upstream URL) unless preserve_host_header; Content-Length dropped (re-encoded chunked); Content-Encoding dropped on response (libcurl auto-decompresses with Accept-Encoding: ""). Runtime extensions: NurlHttpStream grew hdr_buf + header-write callback handling 1xx informational lines; new ABI fns nurl_http_stream_pump_headers / _header_count / _header_name / _header_value (compiler emits decls in the runtime preamble + nurl_sym_def types). Compiler patch: 4 emit declare lines + 4 nurl_sym_def entries in compiler/nurlc.nu. NURL bindings: http_stream_pump_headers / _header_count / _header_name / _header_value in stdlib/ext/http.nu. Limitations (v1): request body via CURLOPT_POSTFIELDS (NUL-truncating; JSON/text fine, binary uploads not yet); response chunks travel through NUL-terminated char* (SSE/JSON/text fine, binary not yet); no Trailer passthrough; no Expect: 100-continue upstream signalling. The buffered ! HttpResponse HttpErr path is intentionally not exposed — multi-field-struct payloads inside ! T E trip a known compiler heap-boxing bug (see http_request.nu:119 ParsedHead workaround note); streaming is the only mode that matters for AI-gateway use cases anyway. Offline test compiler/tests/http_proxy.nu exercises proxy_default_opts, proxy_err_name (3 variants), __is_hop_by_hop (8 hop-by-hop tokens + 2 non-matches), __build_upstream_url (slash-collision normalization + query suffix), __build_request_headers_blob (default-strip + preserve-host paths), __response_header_dropped (8 cases). Carve-out added to run_tests.sh + run_tests.bat.stdlib/ext/http_server.nu server_run_once accepts ONE connection then drives a __serve_keepalive_loop that walks request → handler → response repeatedly until peer close, idle timeout, explicit Connection: close (request OR response), or max_keepalive_requests cap (default 1000) fires. Policy: HTTP/1.1 + no Connection: close → reuse; HTTP/1.0 → always close (no opt-in for Connection: keep-alive in v1). New server_default_max_keepalive_requests → 1000 and server_new_full TcpListener handler i idle_ms i max_keepalive_reqs → HttpServer for explicit override. Pre-existing tcp_set_timeout (per-recv) doubles as the idle deadline for the wait between requests on a persistent connection. Per-request write errors close the connection but no longer bubble out of server_run_once. Bench: 100 sequential /api/health requests dropped from 5152 ms (close-each-time) to 136 ms (reuse) — ~38× speedup against the canonical examples/static_server.nu.getaddrinfo).Channel[T]): Evolve the current i64 channels into fully generic ones (depends on closure-shaped generic propagation).nurl_signal_install_shutdown covers SIGINT/SIGTERM (POSIX) and Ctrl+C/Break/Close (Win32). Generic nurl_signal for arbitrary handlers (logging on SIGUSR1, etc.) is still TODO.-lcurl: Allow pure-server builds to omit libcurl when client functionality is unused.claude_stream_event_input_json_delta SseEvent → ? String (owned partial_json from content_block_delta whose delta.type == "input_json_delta") plus the surrounding event-classification surface needed to drive a streaming chat-with-tools UI from a parsed SseEvent: claude_stream_event_index (block index from content_block_start/content_block_delta/content_block_stop), claude_stream_event_block_kind (text/tool_use/thinking from content_block_start), claude_stream_event_tool_use_id + claude_stream_event_tool_use_name (tool_use opener metadata), claude_stream_event_stop_reason (message_delta.delta.stop_reason — distinct from the synchronous response field which doesn't appear in the streaming envelope), claude_stream_event_error_type + claude_stream_event_error_message (streaming error frame). All extractors return ? String / ? i and are None for any wrong-shape event so callers chain probes without nested matches. 33-case offline test compiler/tests/anthropic_stream.nu exercises every extractor against synthetic SSE frames including a two-frame partial_json accumulation that round-trips into the complete tool-args JSON. Closes the streaming + tool-use combination — NURL agents can now stream Claude responses token-by-token AND tool-call arguments token-by-token in the same loop.HttpOptions struct: Bundle timeout, redirects, follow_count, verify_tls, user_agent overrides.Vec[Header] for http_request: Type-safe header input (depends on multi-field Option Some-arm).stdlib/ext/mcp_client.nu. McpClient { String endpoint, i timeout_ms } + mcp_call c method ?Json params → ! Json McpErr + convenience wrappers (mcp_initialize, mcp_ping, mcp_tools_list, mcp_tools_call, mcp_prompts_list, mcp_resources_list). McpErr { McpAuth | McpHttp | McpJson | McpProtocol | McpOther }. Server-reported JSON-RPC errors ride the Ok arm; caller probes via mcp_response_is_error / _error_code / _error_message. JSON-RPC id uses now_ms (sufficient uniqueness). HTTP transport only — stdio MCP client deferred until duplex process_run lands.stdlib/runtime.c §16b adds NurlProcChild + 12 ABI calls (nurl_proc_spawn / _err_kind / _pid / _write / _close_stdin / _read_line(timeout_ms) / _read_line_len / _eof / _last_io_err / _wait / _kill / _free); POSIX-full implementation (fork+pipe+execvp + non-blocking poll-driven line reader + CLOEXEC sideband for exec-error reporting + scratch-buffer line accumulator that survives chunk boundaries); Win32 + WASI return ProcessOther stubs. (2) stdlib/std/process.nu exposes ProcChild { s raw } + process_spawn / proc_pid / proc_write_line / proc_read_line(timeout_ms)→? String / proc_close_stdin / proc_eof / proc_kill(sig) / proc_wait / proc_free (free reaps via SIGTERM-with-timeout-then-SIGKILL fallback). (3) stdlib/ext/mcp_stdio.nu provides McpStdioClient { ProcChild child, i default_timeout_ms } + mcp_stdio_spawn / mcp_stdio_call(method, ? Json params, timeout_ms) + mcp_stdio_initialize / mcp_stdio_ping / mcp_stdio_tools_list / mcp_stdio_tools_call / mcp_stdio_prompts_list / mcp_stdio_resources_list. McpStdioErr { McpStdioSpawn | McpStdioIo | McpStdioTimeout | McpStdioEof | McpStdioJson | McpStdioProtocol | McpStdioOther } distinguishes the failure modes the HTTP McpErr couldn't (timeout vs. EOF vs. malformed framing). Read loop matches JSON-RPC id so server-initiated notifications (id-less frames) are auto-skipped without surfacing as the wrong response. Tests: compiler/tests/process_spawn_basic.nu (cat-echo round-trip + missing-cmd + timeout + SIGTERM exit-128+15) and compiler/tests/mcp_stdio_basic.nu (cat-echoes-the-request → mcp_stdio_call returns it because the id round-trips, plus McpStdioSpawn / McpStdioEof error mapping). Closes the LLM-host symmetry: NURL can now consume any MCP server regardless of whether it ships as HTTP or stdio.prompts/list, prompts/get, resources/list, resources/read.Mcp-Session-Id header for stateful sessions, JSON-RPC batch requests, Authorization (Bearer) middleware.Making NURL easier to write and debug.
nurlfmt): A deterministic, opinionated formatter for NURL source.tooling/vscode-nurl (currently syntax-only) to the LSP.bench/ for compiler self-host and stdlib hot-paths.examples/ (small JSON pretty-printer, wc/grep/cat clones, MCP client demo, agent loop variants).response_set_header dedup — shipped 2026-05-07: response_set_header in stdlib/ext/http_response.nu now REPLACES any existing header with the same name (case-insensitive ASCII compare per RFC 7230 §3.2) instead of always appending. New companion response_add_header always pushes a fresh entry — for Set-Cookie (RFC 6265 §3) and WWW-Authenticate challenges. response_set_cookie in stdlib/ext/http_auth.nu switched to response_add_header so multiple cookies still produce multiple Set-Cookie: lines. Regression cases dedup_set_header + add_header_repeats added to compiler/tests/http_response_builder.nu. Why this matters: previously, building a JSON response off response_text (which sets Content-Type: text/plain) and then overriding via response_set_header r Content-Type ... produced TWO Content-Type: headers on the wire — silent bug since most clients use the first one. Hit while writing examples/static_server.nu's /api/health handler.examples/static_server.nu — single-file template that composes the full HTTP stack: tcp_listen + server_new + server_run (Phase 1/4), router_get with / + /*path (Phase 6), serve_static + mime_for_ext + ..-rejection (Phase 7), with_access_log + with_metrics + metrics_handler Prometheus exposition + with_cors_default (Phase 8 / Phase 6 helper), signal_install_shutdown for Ctrl+C / SIGTERM. Auto-creates ./public/index.html on first boot, exposes /, /*path, /api/health and /metrics. Verified end-to-end against curl: 200 on a hit (HTML + correct Content-Type from extension), JSON on /api/health, Prometheus text-exposition on /metrics, 404 on a miss, 403 on ..-traversal, access-log line per request, metric counters increment correctly.docs/GOTCHAS.md — 10 active compiler quirks with quick-ref table + per-item symptom/why/workaround/real-source-pointer. Covers the five originally listed (a &/| arity, b multi-field struct mutation through closures, c mutable enum binding miscompile, d vec_get [MultiFieldStruct] default, e bare @-fn closure coercion) plus ! T E Ok-arm width, same-line shadowing, parens-required calls, ternary arity cascades, vec_clone absence. Linked from README.md "Known Limitations".Scaling NURL for team use and external integration.
docs/spec.md) and ownership/auto-drop guidelines (docs/MEMORY.md).Last updated: May 7, 2026 — Phase 5.4 keep-alive shipped (HTTP/1.1 persistent connections, ~38× speedup on hot endpoints). Phase 7 closed out with parse_form_urlencoded + request_form_pairs (HTTP form bodies). Phase 8 mostly shipped (access log, Prometheus metrics, per-conn idle timeout, graceful-shutdown via SIGINT/SIGTERM/Ctrl+C). Production-ready for typical workloads; remaining hardening: per-request total timeout, configurable parser limits. docs/GOTCHAS.md shipped — 10 compiler quirks indexed for future contributors and LLM-driven authoring. Canonical static-file server demo (examples/static_server.nu) shipped — single-file template that composes the full Phase 1/4/5.4/6/7/8 HTTP stack into a working server. response_set_header dedup semantics shipped — response_text + response_set_header r Content-Type ... now produces ONE header on the wire instead of two. Anthropic streaming SSE-event extractor surface shipped — claude_stream_event_input_json_delta plus 7 companions (index, block_kind, tool_use id/name, stop_reason, error type/message); NURL streaming chat UIs can now deliver text AND tool-call arguments token-by-token in the same loop. MCP stdio client shipped — stdlib/ext/mcp_stdio.nu over a new duplex-stdio runtime (nurl_proc_spawn + 11 companion calls, POSIX-full / Win32 stub) with process_spawn / proc_read_line(timeout_ms) / proc_write_line / proc_kill wrappers in stdlib/std/process.nu; closes the LLM-host symmetry — NURL hosts can now consume MCP servers shipped as either HTTP or stdio binaries.*