how alten
is built.

alten is an AI copilot for older adults browsing the web. You tap a smiley buddy, say what you want to do in your own language, and alten points at the exact button to press, says the step out loud, and waits for you. It stops you before scam links, and it gives your family a calm dashboard of how the week went.

01Overview

Getting stuck online is rarely about not being able to click. It's about not knowing which thing to click, being afraid of pressing the wrong one, and having nobody to ask. The usual fixes all have a catch: a grandkid on the phone isn't always free, remote-desktop tools take control away from the person, and AI agents that do the task for you teach nothing and can be steered into paying the wrong site.

alten takes the other path: the person stays in control, and alten only points. It never clicks, types, or sends anything. The one thing it does itself is open a website the person names.

The product has three surfaces:

• The extension (for the senior): a buddy in the corner of every page. Tap it, type or talk, and it guides one step at a time with a pointer, a 2–7 word caption, and a spoken line.
• The family dashboard (for the family): pair with a 6-digit code, then see a weekly recap, every help session step by step, scam alerts, one-tap shortcuts you set up, and voice/language settings.
• alten.select: one domain for the landing page, the dashboard, this documentation, the API, and email.

02Prize tracks at a glance

Each card links to the section with the full technical detail and the files that prove it.

03Architecture

Everything the senior touches runs inside their own Chrome. The extension reads the page locally, asks the planner model for one next step, and talks to ElevenLabs for voice. It reports help sessions, scam warnings and heartbeats to the dashboard API, which stores them in Tiger Cloud. The family reaches everything through alten.select.

04The Chrome extension

A Manifest V3 extension: a service worker (background.js) that owns every network call, plus content scripts injected on all http(s) pages. The heavy lifting is local JavaScript; the model only ever sees a compact text description of the page.

4.1 One step, end to end

1. Ask. The person taps the buddy (or presses the listen shortcut) and speaks, or types in the tray. Speech goes to ElevenLabs Scribe (§6).
2. Shortcut the obvious. If the goal names a site ("go to cbc.ca") and we're not on it, namedSiteUrl() opens it directly. That's the only action alten takes for the person.
3. Scan. scanner.js builds the page model: URL, title, focused field, filled fields, open popups, and a numbered PICK ONE list of controls (§4.2).
4. Plan. The service worker sends goal + history + page model to the planner and gets back exactly one JSON action (§5). remapTarget() re-resolves the target from the words in the instruction itself (preferring controls inside an open popup, breaking ties between same-named controls, and refusing Close/X on a login gate), so a stale or off-by-one index can't point at the wrong button.
5. Guard. Before pointing at a link or opening a URL, judgeTarget() runs the scam checks. A hit replaces the pointer with "Stop. That looks like a scam. Don't tap it." (spoken) and notifies the family (§4.3).
6. Point and speak. The buddy flies to the control, the highlighter outlines it, a caption names it in quotes ("Tap “Compose”."), and the line is spoken with ElevenLabs.
7. Wait for the human. watchClick() / watchField() listen for the person's own click or typing, including inside same-origin iframes. Big steps get a Yes/No first; open choices (sign-in method, size, seat) get choose, which points at the options and lets them pick.
8. Check and repeat. Re-scan. looksDone() checks the page for real outcomes ("message sent", "added to cart", a booking confirmation) so alten stops when the job is done, not when a form merely appears.

4.2 Page scanner

Real websites are hostile to automation: overlays, cookie walls, icon-only buttons, calendars with 60 identical "12"s, shops that render menus in iframes. The scanner's job is to hand the model a list it can't misread.

• What counts as a control: links, buttons, ARIA roles (menuitem, tab, option, switch, combobox, gridcell…), inputs, contenteditable, [onclick], focusable elements, and calendar day cells ([data-date]).
• Only what the person can see and hit: on-screen, non-zero size, not hidden, and topmostAt(), meaning elementFromPoint agrees it isn't covered by something else.
• Popups first: overlayRoots() detects real modals by z-index stacking and viewport coverage. Controls inside them are marked IN-POPUP and listed first; the prompt forbids acting behind an open modal.
• Noise filters: cookie/consent banners and "sign in to save" nags are dropped unless the goal is about them; on a real login gate, Close/X is hidden so the model can't dismiss it.
• Labels with context: generic labels ("Add", "Select", "More") get context from the row or card (contextOf(), dishNear(), monthNear()), so the model sees "Add · Iced Latte $5.10", not five identical "Add"s.
• Ranking: scoreItem() weights controls by action type and overlap with the goal's words; nested duplicates are collapsed with moreSpecific(). No cap: every visible control gets a number.

// What the planner actually receives (trimmed)
GOAL: send an email to my grandson aryan saying happy birthday
ON: mail.example.com · FOCUS: none · FILLED: none
PICK ONE:
0. button "Compose"
1. link "Inbox"
2. row "Sender 1 · Newsletter 1 — weekly update"
…

4.3 Scam guard

scam.js judges any link or URL before alten will point at it or navigate to it. It's deliberately explainable: every warning carries plain-language reasons that end up in the family alert.

• Trusted list: ~50 registrable domains (banks, government, big retailers, mail) are never flagged. "Limited time" on Amazon is shopping, not a scam.
• Lookalikes: the host is folded (0→o, 1→l, 3→e, 5→s, rn→m, vv→w) and compared to the trusted list with an edit distance of ≤ 2, which catches payp4l.com → paypal.com.
• Other signals: links to .exe/.zip/.scr/.apk/.msi downloads, non-ASCII hostnames (IDN homographs), and pressure wording ("verify your account", "gift card", "account locked") on unknown hosts.
• Severity + family control: reasons map to low / medium / high. The family's "scam guard" setting (relaxed / balanced / strict) sets the threshold.
• Reporting: the event sent to the dashboard strips the query string and hash (they can hold tokens), keeping only origin + path.

4.4 Sessions across pages and tabs

Most tasks cross page loads. Every step is persisted to chrome.storage (goal, history, step, pending confirmation, tab id) and resumed by the content script on the next page via continueSession(). stillOnTask() compares sites so an unrelated page doesn't resume an old goal, and gb.return brings the person back to the original tab when a flow opens a new one. A fresh browser start clears the session but keeps pairing and family settings.

05AI planner: Google Gemini → Gemma 4

The planner turns "what they want" plus "what's on screen" into exactly one next action. We built it on Google's models from the first commit.

How it evolved

• v1: the Gemini API. The first extension (commit 1998555) called the Gemini API directly with a gemini-3.5-flash → gemini-flash-latest → flash-lite fallback chain, a JSON response schema, and a text-only page model. It was verified end to end in headless Chrome against the real Gemini API on our practice pages (see PLAN.md).
• Why it changed: guidance has to feel instant. On the free tier (~15 requests/minute) a multi-step task produced visible "thinking…" pauses between steps.
• Now: Google's Gemma 4 26B. The live planner runs gemma-4-26b, Google's open model built from Gemini research, through Logfare's OpenAI-compatible endpoint, with step-3.7-flash only as an availability fallback.

Request contract

POST https://logfare.ai/v1/chat/completions
{ "model": "gemma-4-26b", "temperature": 0.1, "max_tokens": 180,
  "response_format": { "type": "json_object" },
  "messages": [ system rules, language note, …history, page model ] }

// The only shape the model may answer with
{ "action": "click|type|select|scroll|say|navigate|choose|done",
  "target_index": 0, "url": "", "text": "", "instruction": "",
  "confirm": false, "choices": [], "done": false, "message": "" }

Making a model safe to follow

• Grounded indices: target_index must come from this turn's PICK ONE list; numbers reset every turn and the client re-resolves them by name (remapTarget()).
• Short, speakable instructions: 2–7 words, the control named in quotes, never an index, never an explanation. They're read aloud, so brevity is accessibility.
• Hard rules in the prompt: never type passwords; never pick a sign-in method, size or payment for the person unless the goal named it; only navigate to sites they named; work only inside an open popup; done only when the page shows the outcome.
• Latency budget: a 5.5 s abort per call, 429 back-off (two retries), one retry on 500/502/504, then fall through to the next model. Malformed or fenced JSON is recovered by extractJson().
• Language: altenLanguageNote() adds a system message: answer in the person's language (auto) or the family's pick, while keeping on-screen button names exact (§6).
• Privacy by design: no screenshots, ever. The model sees a text list of controls, not pixels, which is cheaper, faster, and far less revealing.

06Voice: ElevenLabs

For a lot of older adults, reading small UI text is the hard part. Voice isn't a feature bolted on; it's the main channel, in both directions, in the person's own language. We use three ElevenLabs APIs.

Speech in: Scribe (scribe_v1)

• Our own end-of-speech detection. A Web Audio AnalyserNode computes RMS energy; recording stops after 700 ms of quiet once words were heard, with a 3.2 s grace period to start talking and a 12 s hard cap. Older speakers pause; this doesn't cut them off mid-thought.
• The MediaRecorder webm blob is posted to /v1/speech-to-text. If the family pinned a listening language we send language_code; on auto we omit it so Scribe detects whatever they speak.
• Fallback: the browser's SpeechRecognition keeps voice working if the API is unavailable.

Speech out: Multilingual v2 (eleven_multilingual_v2)

POST /v1/text-to-speech/{voice_id}?output_format=mp3_44100_64
{ "model_id": "eleven_multilingual_v2",
  "voice_settings": { "stability": 0.32, "similarity_boost": 0.78, "style": 0.42,
                      "use_speaker_boost": true, "speed": 0.85 } }

• Tuned for warmth over flatness: lower stability and moderate style give a conversational read; speaker boost adds presence on laptop speakers. The default voice is Sarah (soft, warm).
• Family-controlled speed: the dashboard slider (0.5–1.5×, default 0.85) is clamped to ElevenLabs' 0.7–1.2 range at request time.
• Low latency path: the service worker calls TTS outside the page's context, returns a small 64 kbps mp3 as base64, and the page plays it immediately. Every caption and every scam warning is spoken.

The voice library, in the family dashboard

• GET /dashboard/api/voices proxies ElevenLabs GET /v1/voices (premade + any custom or cloned voices) with an in-memory TTL cache, returning name, description, category and preview_url, so family can audition voices before picking one.
• If the key can't list voices (no voices_read scope), the route degrades to a curated set of six premade voices instead of failing.
• The chosen voice_id and speed sync to the extension about once a minute (alten-api.js), so a change on the dashboard is heard on Nana's laptop without touching it.

One pipeline, many languages

Settings have two independent fields: speak_language (what alten says; default auto) and listen_language (what Scribe expects; defaults to the same). With auto, a question asked in Spanish is transcribed by Scribe, answered by the planner in Spanish, and spoken by Multilingual v2 in Spanish, while button names stay exactly as they appear on screen so the person can still find them. Our demo shows a coffee order asked as "pídeme un café con leche helado grande" and guided with "Haz clic en “Large”".

07Data: Tiger Data

Everything alten records is an event in time: a question, a step, a click, a scam warning, a heartbeat. So the whole backend is designed around TimescaleDB on Tiger Cloud rather than treating it as plain Postgres.

Schema

Continuous aggregates, in real time

CREATE MATERIALIZED VIEW weekly_help
WITH (timescaledb.continuous, timescaledb.materialized_only = false) AS
SELECT time_bucket('1 week', time) AS week, senior_id,
       count(*) FILTER (WHERE event_type = 'question')  AS questions,
       count(*) FILTER (WHERE event_type = 'completed') AS completed,
       count(*) FILTER (WHERE event_type = 'abandoned') AS abandoned
FROM help_events GROUP BY week, senior_id WITH NO DATA;

SELECT add_continuous_aggregate_policy('weekly_help',
  start_offset => INTERVAL '8 weeks', end_offset => INTERVAL '1 minute',
  schedule_interval => INTERVAL '5 minutes');

• Three aggregates feed the weekly recap: weekly_help (asked / finished solo / gave up), weekly_scams (by severity), and daily_domains (top sites, counting one row per question so a five-step session counts once).
• Real-time aggregation (materialized_only = false) unions materialized buckets with raw rows past the watermark, so a scam blocked 10 seconds ago is already in "scams blocked" on the recap during a live demo.
• Week-over-week in one query: the recap uses time_bucket('1 week', now()) and the bucket before it to compute the "+7 vs last week" deltas straight from the aggregates.
• A lesson we paid for: refreshing an aggregate up to now() materializes the current, incomplete bucket, and real-time aggregation then stops adding newer rows to it, so the live recap froze. Our seed and tooling now refresh complete buckets only.

A rate limiter on a hypertable

Pairing and linking are brute-forceable (a 6-digit code), so every attempt is one row in api_attempts with a sha256 of the IP or device, never the raw value. One statement inserts and counts:

WITH ins AS (INSERT INTO api_attempts (bucket, key) VALUES ($1, $2))
SELECT count(*)::int AS n FROM api_attempts
WHERE bucket = $1 AND key = $2
  AND time > now() - make_interval(mins => $3);

Because it lives in the database rather than memory, the limit holds across every serverless instance. An index on (bucket, key, time DESC) keeps the lookback cheap, and the 1-day retention policy means cleanup is a chunk drop instead of a DELETE.

Operations

• Idempotent migrations (IF NOT EXISTS, if_not_exists => true) tracked in schema_migrations; Vercel runs them before every build (vercel-build: tsx db/migrate.ts && next build), so the schema can't drift from the code.
• The postgres driver with prepare: false for pooled connections; the connection string exists only in server environment variables.
• Local development mirrors production with the timescale/timescaledb:latest-pg17 Docker image, handy when a venue network blocks outbound database ports.

08Family dashboard & API

A Next.js 16 App Router app served under basePath: "/dashboard". API routes return { ok, data } or { ok, error } through one wrapper that turns zod, auth and JSON errors into clean responses.

Scam alert emails

Medium and high scam events trigger a branded email to each linked family member through Resend's batch API, sent after the response with Next's after() so email can never slow or fail ingest. Each email is personal (own unsubscribe link), carries List-Unsubscribe + List-Unsubscribe-Post headers and an idempotency key, and a site already emailed about in the last hour is skipped so one page tripping the guard ten times sends one email.

09Domain: alten.select

.select is a top-level domain operated by GoDaddy Registry. IANA lists its registry operator as Registry Services, LLC, with GoDaddy Registry as administrative and technical contact. We registered alten.select through Porkbun, and the name is the pitch: alten helps you select the right thing on the page.

One domain carries the entire product:

// landing page v1/vercel.json: multi-zone routing on one domain
{ "rewrites": [
  { "source": "/dashboard/:path*", "destination": "https://<dashboard-project>/dashboard/:path*?__via=alten" },
  { "source": "/api/:path*",       "destination": "https://<dashboard-project>/dashboard/api/:path*?__via=alten" } ] }

alten.select is the only public address. The landing project redirects its own Vercel host and www to the apex, and the dashboard's proxy.ts redirects any page opened directly on its Vercel host to alten.select/dashboard. Requests forwarded by alten.select carry a __via marker so they are never bounced, and API routes are exempt so extensions installed before the switch keep working.

Because the dashboard lives under a path on the same origin rather than a subdomain, the family's session cookie, the email links, the unsubscribe page and the docs all share one trustworthy address, the one that's printed on every email and spoken by nobody but alten.

10UI/UX

We designed for a specific person: 78, comfortable with email, nervous about everything else, reading glasses somewhere nearby. Every decision below traces back to her.

The extension

• One step at a time. Never a list of instructions. The next step appears only after the person finishes this one.
• Show, then say. The buddy physically flies to the button, the button gets a thick outline, and a 2–7 word caption names it in quotes, all spoken aloud.
• The person does the clicking. Nothing is paid, sent or submitted unless they do it, which is both safer and how people actually learn.
• Choices stay theirs. Sign-in method, size, seat, payment: alten points at the options and waits instead of guessing.
• Calm under pressure. Scam warnings interrupt with the same friendly voice and a short, direct sentence rather than a scary modal.
• A character, not a chatbot. The lime smiley is the same face on every page, in the dashboard, in emails and on the landing page, with moods (happy, wary, sleepy, thinking, dizzy) that match what's happening.
• Family-tunable: highlight size (normal / large / extra large), voice, speed and language, all set remotely.

The family dashboard

• A tiny desktop. Each feature is a "file" with a window: nanas-week.png (recap), activity.log, alerts.app, shortcuts/, settings.cfg. One panel at a time, the recap open on arrival.
• The recap is a shareable image, rendered from the live card with html-to-image, so a week of progress can go straight into the family group chat.
• Glanceable status in the menu bar: online, current site, last help, all-good or scam-blocked.
• Plain, warm copy ("nana got it in 3 steps ^ω^") and kaomoji empty states instead of charts full of zeros.
• Container queries keep the recap readable from phone to widescreen; light and dark themes.

Accessibility across the product

• Semantic landmarks, labelled controls, aria-live status for the demo, visible focus rings everywhere.
• prefers-reduced-motion turns off animation and video autoplay on the landing page.
• Draggable hero items also move with arrow keys; tables and carousels are keyboard reachable.
• Voice as a first-class path: nothing requires reading small text.

11Security & privacy

• Device auth: the device token is generated once and only a sha256 hash is stored server-side; the raw token never leaves the laptop.
• Family sessions: HMAC-signed, httpOnly, SameSite=Lax cookies; every family route checks the link between that family member and the senior.
• Pairing: 6-digit codes expire after 24 hours; link attempts are rate limited per IP and device via the Tiger hypertable above.
• Validation: every request body is parsed with zod; batched events are accepted partially so one bad event can't drop a whole batch.
• Data minimization: no screenshots; URLs are stored without query strings; passwords are never typed or read; heartbeats expire after 7 days.
• Unsubscribe links are HMACs scoped to their purpose, so they can never be replayed as a login.

12Testing & tooling

• Real browser, real extension: Puppeteer drives Chrome for Testing with the unpacked extension loaded, pairs it with a local dashboard, types a goal into the buddy, and follows the pointer like a person would.
• Practice pages reproduce the hard parts of real sites (Gmail-style compose, a coffee shop, a cart, a cinema flow with a login modal, dual-month calendars, a fake "account locked" page) so regressions show up before a demo.
• The landing page's demo videos are real recordings from that harness (page.screencast → ffmpeg, H.264 with faststart), not mockups.
• Type-checking and ESLint on the dashboard; npm run email:preview renders or sends a sample alert email; db:check prints the live Timescale setup (hypertables, aggregates, policies) to confirm a migration.

13Stack reference