{"authors":[{"name":"Xiyan Shao","url":"https://www.xiyan.dev/"}],"description":"Xiyan Shao's personal blog. I work on Gemini's coding and agent capabilities at Google DeepMind, and write about a fairly random mix of things here.","feed_url":"https://www.xiyan.dev/feed.json","home_page_url":"https://www.xiyan.dev/","items":[{"content_html":"\u003cp\u003eThere\u0026rsquo;s a small Tetris board on the home page of this blog, playing itself. The caption offers \u0026ldquo;take a turn →\u0026rdquo;, and if you click it the bot hands you the keyboard; press escape and it takes over again, calmly fixing whatever mess you left. (The same widget haunts the 404 page, where it has more room to brood.)\u003c/p\u003e\n\u003cp\u003eThis post is about what\u0026rsquo;s inside it: a guideline-accurate Tetris engine in Rust, a search bot that out-digs Cold Clear 2 in the one comparison I could make rigorous, a neural network that did not survive its own benchmark, and an architecture for running tree search inside a 60 Hz frame budget without dropping a frame. The whole thing arrives in your browser as 114 KB of WebAssembly.\u003c/p\u003e\n\u003cfigure class=\"tetris-inline\" id=\"tetris-inline-0\" aria-label=\"An AI playing Tetris\"\u003e\n \u003cdiv class=\"tetris-inline__board\" id=\"tetris-inline-0-board\"\u003e\u003c/div\u003e\n \u003cfigcaption class=\"tetris-inline__note\"\u003e\n \u003cspan class=\"tetris-inline__ai\"\u003ean AI plays. \u003cbutton type=\"button\" class=\"tetris-inline__cta\" id=\"tetris-inline-0-cta\"\u003etake a turn\u003cspan aria-hidden=\"true\"\u003e \u0026rarr;\u003c/span\u003e\u003c/button\u003e\u003c/span\u003e\n \u003cspan class=\"tetris-inline__playing\" hidden\u003e\u003ckbd\u003e\u0026uarr;\u003c/kbd\u003e rotate \u0026middot; \u003ckbd\u003eshift\u003c/kbd\u003e hold \u0026middot; \u003ckbd\u003eesc\u003c/kbd\u003e release\u003c/span\u003e\n \u003c/figcaption\u003e\n\u003c/figure\u003e\n\u003cstyle\u003e\n \n .tetris-inline { display: none; }\n .tetris-inline[data-ready] {\n display: flex;\n flex-direction: column;\n align-items: center;\n gap: 0.6rem;\n margin: 2.5rem auto;\n }\n .tetris-inline__board {\n \n width: 220px;\n aspect-ratio: 10 / 22;\n line-height: 0;\n color: var(--color-text-secondary);\n }\n .tetris-inline__board canvas { display: block; }\n .tetris-inline__note {\n margin: 0;\n text-align: center;\n font-size: var(--font-size-sm, 0.85rem);\n line-height: 1.4;\n color: var(--color-text-tertiary, var(--color-text-secondary));\n }\n .tetris-inline__playing {\n white-space: nowrap;\n font-size: 0.75rem;\n }\n .tetris-inline__cta {\n font: inherit;\n color: var(--color-link, var(--color-accent, currentColor));\n background: none;\n border: 0;\n padding: 0;\n cursor: pointer;\n text-underline-offset: 2px;\n text-decoration: underline;\n text-decoration-thickness: 1px;\n text-decoration-color: color-mix(in oklch, currentColor 35%, transparent);\n transition: text-decoration-color var(--duration-fast, 150ms) var(--easing-standard, ease);\n }\n .tetris-inline__cta:hover,\n .tetris-inline__cta:focus-visible { text-decoration-color: currentColor; }\n .tetris-inline__cta:focus-visible {\n outline: 2px solid var(--color-focus-ring, var(--color-accent, currentColor));\n outline-offset: 3px;\n border-radius: 2px;\n }\n \n @media (hover: none) {\n .tetris-inline__cta { display: none; }\n }\n .tetris-inline__playing kbd {\n font: inherit;\n padding: 0 0.2em;\n border: 1px solid color-mix(in oklch, currentColor 25%, transparent);\n border-radius: 3px;\n }\n\u003c/style\u003e\n\n\n\n\n\n\u003cscript type=\"module\"\u003e\n const MODULE = \"\\/tetris-embed\\/tetris-embed.js?v=c14d3ae6\";\n const GLUE = \"\\/tetris-embed\\/tetr_embed.js?v=1fd0b005\";\n\n const fig = document.getElementById(\"tetris-inline-0\");\n const host = document.getElementById(\"tetris-inline-0-board\");\n const cta = document.getElementById(\"tetris-inline-0-cta\");\n const aiNote = fig.querySelector(\".tetris-inline__ai\");\n const playingNote = fig.querySelector(\".tetris-inline__playing\");\n\n \n \n const CAN_TAKEOVER = window.matchMedia(\"(hover: hover) and (pointer: fine)\").matches;\n\n \n \n function setCaption(state) {\n const playing = state === \"human\";\n if (aiNote) aiNote.hidden = playing;\n if (playingNote) playingNote.hidden = !playing;\n }\n\n let handle = null;\n let watchdog = 0;\n\n \n \n async function show() {\n if (handle) return;\n fig.dataset.ready = \"\";\n try {\n const { mount } = await import(MODULE);\n if (handle || !(\"ready\" in fig.dataset)) return;\n handle = mount(host, {\n theme: { board: { radius: 0, background: \"transparent\", frame: null } },\n fullWidth: true,\n monochromeWhileAI: true,\n sidebar: false,\n interactive: CAN_TAKEOVER,\n showHint: false,\n glueUrl: GLUE,\n label: \"Tetris — an AI is playing; click to take over, Esc to release\",\n onControlChange: setCaption,\n });\n clearTimeout(watchdog);\n watchdog = setTimeout(() =\u003e { if (!host.querySelector(\"canvas\")) hide(); }, 8000);\n } catch (error) {\n console.error(\"tetris shortcode failed to load:\", error);\n hide();\n }\n }\n function hide() {\n clearTimeout(watchdog);\n handle?.destroy();\n handle = null;\n delete fig.dataset.ready;\n }\n\n cta?.addEventListener(\"click\", () =\u003e {\n const canvas = host.querySelector(\"canvas\");\n if (canvas) { canvas.focus(); canvas.dispatchEvent(new PointerEvent(\"pointerdown\", { bubbles: true })); }\n });\n\n show();\n\u003c/script\u003e\n\n\u003ch2 id=\"three-weekends-in-2023-then-silence\"\u003e\n \u003cspan\u003eThree weekends in 2023, then silence\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#three-weekends-in-2023-then-silence\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eIn March 2023 I wanted to learn \n\n\n\n\n\n\u003ca href=\"https://bevyengine.org/\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003eBevy\u003c/a\u003e\n, and a Tetris clone seemed like a good excuse. The early commit log is honest about how that went: \u0026ldquo;fix: ghost block glitch\u0026rdquo;, \u0026ldquo;fix: wall kicks (untested)\u0026rdquo;, \u0026ldquo;fix: score and some sounds\u0026rdquo;. Twelve commits, a wasm deployment on April Fools\u0026rsquo; Day, and then nothing. The commit histogram has a three-year hole in it.\u003c/p\u003e\n\u003cp\u003eWhen I came back in April 2026, the first commit was a four-major-version Bevy upgrade. The second thing I did was more telling: instead of adding features, I started pulling every game rule out of the ECS into a module that knows nothing about rendering. Almost everything I ended up caring about followed from that one move.\u003c/p\u003e\n\u003ch2 id=\"the-engine-is-a-function\"\u003e\n \u003cspan\u003eThe engine is a function\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#the-engine-is-a-function\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eThe core crate, \u003ccode\u003etetr-core\u003c/code\u003e, is just under fourteen thousand lines of Rust with zero Bevy types in it. The engine is a pure function: \u003ccode\u003e(seed, input frames) -\u0026gt; (state, events)\u003c/code\u003e. It never reads a wall clock, and every random draw comes from one seeded RNG. Run the same seed with the same inputs and you get a byte-identical game, on macOS, on Linux, or inside a browser.\u003c/p\u003e\n\u003cp\u003eThat purity is the load-bearing decision of the whole project. It makes replays trivial and lockstep multiplayer possible someday, and it made the part that actually mattered cheap: headless AI evaluation, thousands of games in a benchmark harness with no window, no frame loop, and no flakiness.\u003c/p\u003e\n\u003cp\u003eThe rules themselves are guideline Tetris, taken seriously: SRS rotation with the full five-test wall kicks, a seeded 7-bag randomizer, hold, Back-to-Back, combos, and three lock-down modes. The acceptance tests are named after the spec they enforce (\u003ccode\u003eacceptance_kick.rs\u003c/code\u003e, \u003ccode\u003eacceptance_seven_bag.rs\u003c/code\u003e, \u003ccode\u003eacceptance_t_spin.rs\u003c/code\u003e), and they cover the corners people usually skip, like the guideline §7.5 exception where a T piece kicked into a slot by the fifth wall-kick test earns a full T-spin rather than a mini.\u003c/p\u003e\n\u003cp\u003eOne more rule, self-imposed: the AI plays through the exact same input surface as a human. It emits key presses with DAS timing; it cannot teleport pieces. Whatever it achieves, it achieves under the same physics you do.\u003c/p\u003e\n\u003ch2 id=\"teaching-it-to-play\"\u003e\n \u003cspan\u003eTeaching it to play\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#teaching-it-to-play\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eThe bot is an evaluator plus a search, and the seam between them is the most useful idea I borrowed from \n\n\n\n\n\n\u003ca href=\"https://github.com/MinusKelvin/cold-clear-2\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003eCold Clear\u003c/a\u003e\n: scores split into a \u003cem\u003eValue\u003c/em\u003e (how good is this resting board: holes, transitions, wells, height) and a \u003cem\u003eReward\u003c/em\u003e (what did this particular move earn: clear type, spin, Back-to-Back), with combo and B2B state threaded along the search path so rewards stay honest in chains.\u003c/p\u003e\n\u003cp\u003eTwo hand-written evaluators sit behind that interface: the classic Dellacherie-Thiery linear features, and later a verbatim port of Cold Clear 2\u0026rsquo;s freestyle evaluation with its published weights. Three searches sit on the other side: a one-ply greedy, a batch beam, and a best-first graph search with a transposition table. A unit test pins the beam at depth 1 to reproduce greedy exactly, so the tiers can\u0026rsquo;t silently drift apart.\u003c/p\u003e\n\u003cp\u003eThe finding I didn\u0026rsquo;t expect: the evaluator sets the ceiling, and the search only decides how efficiently you reach it. Best-first scales smoothly with node budget right up to the eval\u0026rsquo;s optimum, around 0.67 attack per piece with the tuned weights, and then goes flat. Searching to depth 8 actually scored \u003cem\u003ebelow\u003c/em\u003e depth 6 with a wider beam. Every \u0026ldquo;stronger search\u0026rdquo; experiment converged to the same number; the number only moved when I changed what the evaluator wanted.\u003c/p\u003e\n\u003ch2 id=\"benchmarking-against-a-real-opponent\"\u003e\n \u003cspan\u003eBenchmarking against a real opponent\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#benchmarking-against-a-real-opponent\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eI wanted to know how the bot compared to \n\n\n\n\n\n\u003ca href=\"https://github.com/MinusKelvin/cold-clear-2\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003eCold Clear 2\u003c/a\u003e\n, MinusKelvin\u0026rsquo;s open-source bot and the strongest one I could actually run. CC2 publishes no performance numbers, so I had to generate the baseline myself: a referee that speaks the Tetris Bot Protocol to the CC2 binary over stdio, feeds both bots the same seeded bag, and scores everything with my engine\u0026rsquo;s attack tables.\u003c/p\u003e\n\u003cp\u003eThe first metric was attack per piece from an empty board. My bot scored 1.70. CC2 scored 0.69. I would have loved to believe that, and it was nonsense. The metric was gameable: with no incoming garbage, an empty-board APP contest rewards combo-spam, and my bot had learned to farm the meter while CC2 played real B2B and T-spin attack that survives actual pressure. A search will farm any number you let it farm. I threw the metric out.\u003c/p\u003e\n\u003cp\u003eThe honest replacement is a downstack race: bury both bots under nine rows of holey garbage and count pieces until it\u0026rsquo;s clean. Combo-farming doesn\u0026rsquo;t help you there. Over 24 seeds, my bot cleared in 13.6 pieces against CC2\u0026rsquo;s 16.6, roughly 18% better, and giving CC2 four times the thinking time made it slightly \u003cem\u003eworse\u003c/em\u003e, so the gap is a difference in what the two bots want, not in how long they search. CC2 optimizes attack; my evaluator digs cleaner.\u003c/p\u003e\n\u003cp\u003eHead-to-head versus play was the obvious next claim, and I had to refuse it. My naive harness said I won 3-0, but the win was an artifact: the bot protocol has no message for incoming garbage (you can read CC2\u0026rsquo;s message enum and see it), so every garbage injection forces a stop-and-restart that throws away CC2\u0026rsquo;s search tree. Benchmarking a bot you\u0026rsquo;ve lobotomized isn\u0026rsquo;t a result. The fix was to port CC2\u0026rsquo;s evaluator natively into my framework, so both brains run on the same engine, the same search, and the same garbage: fair by construction. The port is faithful: it digs nine rows in about 17 pieces, matching what the real binary did over the protocol. In that arena, my tuned evaluator wins both axes at every search depth I tested, taking 14 of 16 versus games at the settings that ship.\u003c/p\u003e\n\u003ch2 id=\"ten-times-faster-was-the-goal-seven-is-what-i-banked\"\u003e\n \u003cspan\u003eTen times faster was the goal; seven is what I banked\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#ten-times-faster-was-the-goal-seven-is-what-i-banked\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eAt launch, the best-first attack bot needed 115-158 ms per piece. I wanted 10x, and I started the way everyone starts: by guessing. I guessed the evaluator\u0026rsquo;s arithmetic was the cost. Wrong: it measured at 0.4 µs, faster than the supposedly simpler linear eval. I guessed the per-node board clone. Wrong again, about 3% of per-child time. Two confident refactor plans died in the profiler before I wrote any code.\u003c/p\u003e\n\u003cp\u003eWhat the criterion benchmarks and macOS \u003ccode\u003esample\u003c/code\u003e actually showed: line-clear detection re-scanning the whole board on every lock (5-24 µs, scaling with stack height), the standard library\u0026rsquo;s SipHash dominating the transposition table, and collision checks crawling through a cell grid during move generation. The cure for all three was the same data structure: a bitboard. The search state became a \u003ccode\u003eCopy\u003c/code\u003e struct with one \u003ccode\u003eu64\u003c/code\u003e per column, so forking a node is a stack copy with no allocation, full rows fall out of an AND-fold, and collision is a single bit test. Move generation was made generic over an \u003ccode\u003eOccupancy\u003c/code\u003e trait so it runs directly on bits while the SRS kick tables stay in the engine, never re-encoded, so the search literally cannot disagree with the game about rotation.\u003c/p\u003e\n\u003cp\u003eEvery change had to pass one gate: attack-per-piece numbers byte-identical before and after. The results, per scenario:\u003c/p\u003e\n\u003ctable\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth style=\"text-align: left\"\u003eBoard\u003c/th\u003e\n \u003cth style=\"text-align: left\"\u003eBefore\u003c/th\u003e\n \u003cth style=\"text-align: left\"\u003eAfter\u003c/th\u003e\n \u003cth style=\"text-align: left\"\u003eSpeedup\u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd style=\"text-align: left\"\u003eClean\u003c/td\u003e\n \u003ctd style=\"text-align: left\"\u003e115 ms/piece\u003c/td\u003e\n \u003ctd style=\"text-align: left\"\u003e25 ms/piece\u003c/td\u003e\n \u003ctd style=\"text-align: left\"\u003e4.6x\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"text-align: left\"\u003eNine rows of cheese\u003c/td\u003e\n \u003ctd style=\"text-align: left\"\u003e156 ms/piece\u003c/td\u003e\n \u003ctd style=\"text-align: left\"\u003e22 ms/piece\u003c/td\u003e\n \u003ctd style=\"text-align: left\"\u003e7.1x\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"text-align: left\"\u003eHeavy faucet garbage\u003c/td\u003e\n \u003ctd style=\"text-align: left\"\u003e158 ms/piece\u003c/td\u003e\n \u003ctd style=\"text-align: left\"\u003e22 ms/piece\u003c/td\u003e\n \u003ctd style=\"text-align: left\"\u003e7.2x\u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eThe differential tests that held the bitboard to the engine\u0026rsquo;s exact behavior caught three subtle landmines, and one of them turned out to be a real engine bug: a row completed entirely inside the hidden buffer zone above the visible field was being scored but never removed. The bitboard work paid for itself in correctness before it paid in speed.\u003c/p\u003e\n\u003cp\u003eThe final 2.2x to a clean 10x would have required SIMD, threads (determinism is load-bearing, and wasm threads are their own adventure), or a cheaper move generator that gives up tucks and T-spins, which is to say the bot\u0026rsquo;s whole point. I declined and banked the 7x.\u003c/p\u003e\n\u003ch2 id=\"who-actually-wrote-this\"\u003e\n \u003cspan\u003eWho actually wrote this\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#who-actually-wrote-this\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eThe 2023 commits are mine, keystroke by keystroke. Most of the 2026 ones are not. I work on AI coding agents during the day, and for the comeback I mostly directed one instead of typing: Claude wrote the bulk of the engine rewrite, the search, the benchmark harness, and the embed, across long sessions where my job was to pick the goals, argue about the design, and read every diff. The commit log gives it away if you know what to look for. A human rarely names a commit \u0026ldquo;extract shared RootKey for stale-run + transposition identity (zero-slop batch 11/N)\u0026rdquo;.\u003c/p\u003e\n\u003cp\u003eThe code generation is table stakes by now. What I enjoyed more was letting the agent run experiments. The benchmark harness reduces a bot configuration to a single number, and a single number is a fitness function, so I pointed an autonomous hill-climbing loop at the evaluator\u0026rsquo;s weights: perturb them, play a batch of headless games, keep the change if the number improves, repeat until the time budget expires. It worked. The climbed weights lifted every scenario in the behavior suite, helped most exactly where play is hardest (a 35% gain under the heaviest garbage pressure), and they are what the bot on the home page plays today. The loop also rediscovered classic tuning failures without being told about them. Climbing on the downstack fitness produced a brilliant digger that then lost versus matches. Climbing on the noisy versus fitness reported a +0.50 margin that came out to exactly +0.00 on held-out seeds. And after I had to kill two multi-hour runs by hand, every climb got a hard time budget baked in.\u003c/p\u003e\n\u003cp\u003eWhat the loop could not do is climb past the evaluator\u0026rsquo;s ceiling. Hill-climbing tunes toward whatever the fitness function can already see, and it stalled at the same 0.67 attack per piece as everything else. Getting further meant replacing the evaluator\u0026rsquo;s eyes entirely, which is where the neural net came in.\u003c/p\u003e\n\u003ch2 id=\"the-value-net-that-didnt-make-it\"\u003e\n \u003cspan\u003eThe value net that didn\u0026rsquo;t make it\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#the-value-net-that-didnt-make-it\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eFor a while the repo contained a neural evaluator: a small MLP trained in JAX, run in the browser through Burn. It failed twice, each time more instructively.\u003c/p\u003e\n\u003cp\u003eThe first failure was the obvious one. I distilled it from the weakest hand evaluator, so it faithfully learned to be exactly that strong. You don\u0026rsquo;t pass the leader by imitating the back of the pack.\u003c/p\u003e\n\u003cp\u003eThe second failure was better. Retrained on games from the strong evaluator, the net had only ever seen the games of a bot that never dies. It had no examples of tall boards leading to death, so it scored dangerous stacks \u003cem\u003ehighly\u003c/em\u003e: in its training data, tall boards were always followed by lucrative digging. The beam search, dutifully maximizing the net\u0026rsquo;s opinion, steered straight into a top-out within fifteen pieces. Switching from hand features to the raw board (a 24x10 grid plus combo and B2B state, 242 inputs) fixed the model\u0026rsquo;s capacity and changed nothing about this, because the problem was distribution shift, not expressiveness: the search drives toward boards the net has never seen, where it extrapolates garbage. Even DAgger-style retraining on the net-bot\u0026rsquo;s own games just made it monotonically more pessimistic: validation error went 12.9, 16.2, 21.9 across iterations.\u003c/p\u003e\n\u003cp\u003eThe conclusion, written up in a postmortem that now lives in the repo where the code used to: offline value regression from one policy\u0026rsquo;s data caps out at that policy. Getting past the hand evaluator needs actual self-play RL, which is a different project. I deleted the entire stack (the trainer, the inference crate, and Burn\u0026rsquo;s dependency tree, which had been most of my wasm size budget) and committed \u003ccode\u003edocs/value-net-postmortem.md\u003c/code\u003e in its place. Best trade of the project.\u003c/p\u003e\n\u003ch2 id=\"a-search-that-fits-in-a-frame\"\u003e\n \u003cspan\u003eA search that fits in a frame\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#a-search-that-fits-in-a-frame\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eAll of this has to run on a browser\u0026rsquo;s main thread. The shipped bot\u0026rsquo;s search took 60-120 ms per decision when run to completion, which is four to seven dropped frames every single piece. Web Workers would mean SharedArrayBuffer and cross-origin isolation headers; for a bot whose whole budget is one human reaction window, that\u0026rsquo;s a lot of machinery to avoid an architectural fix.\u003c/p\u003e\n\u003cp\u003eThe architectural fix is to stop letting the search touch time at all. Work and time are separate currencies. The search session, the \u003ccode\u003eMind\u003c/code\u003e, measures work in nodes and exposes four verbs: \u003ccode\u003ereroot()\u003c/code\u003e to aim at a new position, \u003ccode\u003ethink(quantum)\u003c/code\u003e to spend a fixed slice of nodes, \u003ccode\u003ebest()\u003c/code\u003e to read the current answer at any moment, and \u003ccode\u003enodes_expanded()\u003c/code\u003e to meter it. It never reads a clock. \u003cem\u003eWhere\u003c/em\u003e thinking happens is a venue decision made by the caller: benchmarks run a blocking venue that drains the budget in one call, and the browser runs a sliced venue that spends one 16-node quantum per frame. A controller pumps the venue every poll, buffers the finished decision, and only acts when the bot\u0026rsquo;s human-like reaction delay has elapsed.\u003c/p\u003e\n\u003cp\u003eThe shipped budget is 192 nodes, and the number isn\u0026rsquo;t arbitrary: a 200 ms reaction window at 60 Hz is 12 polls, times 16 nodes per poll on wasm. The budget is exactly the window\u0026rsquo;s capacity, and a test fails if anyone raises the constant, because past that point the bot thinks slower than it reacts, which is a gameplay decision someone should make on purpose rather than by editing an integer.\u003c/p\u003e\n\u003cp\u003eBecause the search spends nodes rather than milliseconds, slicing it changed nothing about its play: same node count, same decisions, byte-identical games, now spread across frames. A release-build browser trace confirmed the feel: the worst main-thread task during play is 24 ms and the p99 is 2.4 ms, where each piece used to cost a 60-120 ms stall.\u003c/p\u003e\n\u003ch2 id=\"114-kilobytes-committed\"\u003e\n \u003cspan\u003e114 kilobytes, committed\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#114-kilobytes-committed\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eThe full game (menus, settings, bloom, the Bevy renderer) is a couple of megabytes of wasm, and the home page loads none of it. The widget is a separate 466-line crate, \u003ccode\u003etetr-embed\u003c/code\u003e, that wraps the engine and the bot behind a wasm-bindgen \u003ccode\u003eGame\u003c/code\u003e: tick it at 60 Hz from \u003ccode\u003erequestAnimationFrame\u003c/code\u003e, feed it key events, read cells from a cached snapshot. A small Preact component draws those cells on a canvas. The payload is 114 KB of WebAssembly and about 44 KB of JavaScript, and the artifacts are committed into the blog\u0026rsquo;s repo, so the blog builds with plain Hugo and no Rust toolchain anywhere in its CI.\u003c/p\u003e\n\u003cp\u003eTwo details I\u0026rsquo;m fond of. Determinism survives the FFI boundary: the same seed, handicap, and tick sequence produce byte-identical games in the embed, which makes browser bugs reproducible in a native test. And every number JavaScript passes in gets sanitized, because of a constraint the source comment states plainly: a panic across the wasm boundary aborts the entire module, \u0026ldquo;taking down every \u003ccode\u003eGame\u003c/code\u003e on the page.\u0026rdquo;\u003c/p\u003e\n\u003cp\u003eThe take-a-turn mechanic costs almost nothing, structurally. The engine doesn\u0026rsquo;t know who\u0026rsquo;s driving it. Autoplay is the AI controller; clicking the button swaps in a keyboard controller fed from DOM key state; escape swaps back. Same engine, same rules, different hands.\u003c/p\u003e\n\u003ch2 id=\"whats-next\"\u003e\n \u003cspan\u003eWhat\u0026rsquo;s next\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#whats-next\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eVersus is the obvious milestone: the engine already has garbage queues, cancellation, and attack tables, and the anytime-search architecture was shaped so a future bot can ponder continuously and re-root when garbage lands, instead of starting from scratch each piece.\u003c/p\u003e\n\u003cp\u003eMeanwhile the bot on the home page plays with a deliberate handicap: a human-ish reaction delay and a small error rate. It is beatable. If you want to try, there\u0026rsquo;s a board at the top of this post, another loitering on the \n\n\n\n\n\n\u003ca href=\"/\"\u003ehome page\u003c/a\u003e\n, and the full game, with the menus and the bloom, lives \n\n\n\n\n\n\u003ca href=\"https://www.xiyan.dev/tetr_online/\"\u003ehere\u003c/a\u003e\n. Source for all of it is on \n\n\n\n\n\n\u003ca href=\"https://github.com/xiyan128/tetr_online\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003eGitHub\u003c/a\u003e\n.\u003c/p\u003e","date_modified":"2026-06-10T11:21:36-07:00","date_published":"2026-06-10T01:00:00-07:00","id":"https://www.xiyan.dev/writings/2026/06/10/the-tetris-on-the-home-page/","summary":"There\u0026rsquo;s a small Tetris board on the home page of this blog, playing itself. The caption offers \u0026ldquo;take a turn →\u0026rdquo;, and if you click it the bot hands you the keyboard; press escape and it takes over again, calmly fixing whatever mess you left. (The same widget haunts the 404 page, where it has more room to brood.)\nThis post is about what\u0026rsquo;s inside it: a guideline-accurate Tetris engine in Rust, a search bot that out-digs Cold Clear 2 in the one comparison I could make rigorous, a neural network that did not survive its own benchmark, and an architecture for running tree search inside a 60 Hz frame budget without dropping a frame. The whole thing arrives in your browser as 114 KB of WebAssembly.\n","tags":["Rust","WebAssembly","Game AI","Tetris"],"title":"The Tetris on the Home Page","url":"https://www.xiyan.dev/writings/2026/06/10/the-tetris-on-the-home-page/"},{"content_html":"\u003cp\u003eReading notes for Graduate Operating Systems at UCSD. The notes are based on the \n\n\n\n\n\n\u003ca href=\"https://amyousterhout.com/cse221-winter24/readings.html\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ereadings\u003c/a\u003e\n and discussions from the course.\u003c/p\u003e\n\u003ch1 id=\"historical-perspective\"\u003e\n \u003cspan\u003eHistorical Perspective\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#historical-perspective\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h1\u003e\n\u003ch2 id=\"the-structure-of-the-the-multiprogramming-system\"\u003e\n \u003cspan\u003eThe Structure of the \u0026ldquo;THE\u0026rsquo;-Multiprogramming System\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#the-structure-of-the-the-multiprogramming-system\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eLayered design: (1) Processes (scheduler) (2) memory; storage\n(paging) (3) I/O for peripheral (4) human operator (5) all of your\napps\u003c/li\u003e\n\u003cli\u003eSynchronization via semaphores: \u003ccode\u003eP\u003c/code\u003e wait, \u003ccode\u003eV\u003c/code\u003e signal; 2 ways to use\nthem (1) mutual exclusion (2) coordination\u003c/li\u003e\n\u003cli\u003eSummary: layered OS design; abstraction of processes; semaphores for\nsynchronization; correctness\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"the-nucleus-of-a-multiprogramming-system\"\u003e\n \u003cspan\u003eThe Nucleus of a Multiprogramming System\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#the-nucleus-of-a-multiprogramming-system\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eGoals: extensibility, flexible OS design\u003c/li\u003e\n\u003cli\u003eNucleus: small, supports multiple OSes\u003c/li\u003e\n\u003cli\u003eComponents: IPC via messages; processes (create, stop, remove;\ninternal vs external); similar to microkernel\u003c/li\u003e\n\u003cli\u003eOSes on top of Nucleus: memory \u0026amp; process management (hierarchical\napproach)\u003c/li\u003e\n\u003cli\u003eEconomy of abstraction: processes and messages\n\u003cul\u003e\n\u003cli\u003eExternal processes (replaced by files today), Clock process\u003c/li\u003e\n\u003cli\u003eSynchronization with messages (buggy processes, dynamic)\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eMessages: overhead, buffer management (partitioned pool); API with\nseparate send/wait and send/wait for message/answer\u003c/li\u003e\n\u003cli\u003eSummary: enable flexible OS design; nucleus with layered OSes;\nsynchronization via message passing; clean abstractions\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"tenex-a-paged-time-sharing-system-for-the-pdp-10\"\u003e\n \u003cspan\u003eTENEX, a Paged Time Sharing System for the PDP-10\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#tenex-a-paged-time-sharing-system-for-the-pdp-10\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eTENEX Goals: virtual memory, good engineering (CLI), easy\ndebugging/modification/maintenance, performance, effective file\nsystem, backwards compatibility\u003c/li\u003e\n\u003cli\u003eVirtual Memory: address spaces (process, OS), memory maps (page\ntable), BBN pager (MMU, TLB)\n\u003cul\u003e\n\u003cli\u003eVirtual Address Translation: virtual address → page table →\nphysical address; with TLB: virtual address → TLB → (hit)\nphysical address or (miss) page table\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eBBN Pager: 9 input bits → 11 output bits; physical memory \u0026gt;\u0026gt;\nvirtual memory\u003c/li\u003e\n\u003cli\u003eSharing: pages of memory, page table entries (direct, indirect,\nshared); copy-on-write\u003c/li\u003e\n\u003cli\u003eBackward Compatibility: library/compatibility layer for binary\ncompatibility; \u003ccode\u003ejsys\u003c/code\u003e instruction for old syscalls\u003c/li\u003e\n\u003cli\u003eScheduling: more tolerant of page faults\u003c/li\u003e\n\u003cli\u003eFile System: hierarchical \u003ccode\u003eC\\dir\\file ext.version\u003c/code\u003e (device,\ndirectory, name, extension, version number)\u003c/li\u003e\n\u003cli\u003eSummary: virtual memory; sharing, copy-on-write; time-sharing\nsystem; backwards compatibility\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"hydra\"\u003e\n \u003cspan\u003eHYDRA\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#hydra\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eGoal: decompose OS into subsystems (file system, scheduler, virtual\nmemory); provide protection between subsystems; user-defined\nsubsystems\u003c/li\u003e\n\u003cli\u003eKey terms:\n\u003cul\u003e\n\u003cli\u003eProtection domains: e.g., process vs kernel; same\nrights/capabilities in one region\u003c/li\u003e\n\u003cli\u003eProtected control transfer: syscall\u003c/li\u003e\n\u003cli\u003eRights augmentation\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eHydra:\n\u003cul\u003e\n\u003cli\u003eResources: objects (type, interface, methods, data, pointers)\u003c/li\u003e\n\u003cli\u003eExecution domain: local name space (LNS), stack frame\u003c/li\u003e\n\u003cli\u003eProtection mechanisms: capability (pointer | rights bits);\nkernel rights (standard), auxiliary rights (user-defined)\u003c/li\u003e\n\u003cli\u003eProcedures:\n\u003ccode\u003ewrite(F, buffer, length) { local rights, local vars }\u003c/code\u003e, F:\ncapability; rights checking; templates for changing rights\n(caller vs callee, protected control transfer, rights\naugmentation)\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eSummary: capability-based protection; nucleus-like system for\nbuilding OSes on top\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch1 id=\"structure\"\u003e\n \u003cspan\u003eStructure\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#structure\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h1\u003e\n\u003ch2 id=\"protection-lampson\"\u003e\n \u003cspan\u003eProtection (Lampson)\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#protection-lampson\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eGoals: abstract model; unify discussion of protection concepts\u003c/li\u003e\n\u003cli\u003eObject: things to protect (Hydra: object, Multics: segments)\u003c/li\u003e\n\u003cli\u003eDomains: equivalent class with same rights (Hydra: LNS, Multics:\nprincipal identifier (user), ring)\u003c/li\u003e\n\u003cli\u003eAttributes: read, write, execute\u003c/li\u003e\n\u003cli\u003eAccess Matrix: rows (capability lists), columns (access control\nlists)\u003c/li\u003e\n\u003cli\u003eSummary: objects, domains, access matrix, ACLs (columns), capability\nlists (rows)\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"multics\"\u003e\n \u003cspan\u003eMultics\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#multics\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eProtection Model Principles: Permission not exclusion; check every\nprocess; design is not secret; least privilege; easy to use\u003c/li\u003e\n\u003cli\u003eSegmented Memory Model: Descriptor mapping to app, file, lib, fs,\netc.; virtual address: \u003ccode\u003esegment number, offset\u003c/code\u003e; Segments + paging;\nspecial registers (cs, ds, ss); Descriptor (capability): addr, RWX,\ncall, ring (rights)\u003c/li\u003e\n\u003cli\u003eProtection in Multics: Check protection efficiently; Login;\nPrinciple identifier (user ID)\u003c/li\u003e\n\u003cli\u003eFile System Protection: Derived from File System; ACLs (had to\nstore, inefficient) → capabilities; \u003ccode\u003eopen(\u0026quot;file.txt\u0026quot;, permissions)\u003c/code\u003e\nreturns file descriptor; \u003ccode\u003eread(fd, buffer, length)\u003c/code\u003e\u003c/li\u003e\n\u003cli\u003eProtected Subsystems (e.g. File System): Gates: entry points; Rings:\ngeneralize user-level vs kernel level; least to most privilege;\n(apps, lib, fs, drivers, core os) use different ring levels\u003c/li\u003e\n\u003cli\u003eSummary: Uniform data protection using ACL; memory access protection\nusing descriptors (capability)\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"unix\"\u003e\n \u003cspan\u003eUnix\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#unix\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eGoals: Easy to use, general purpose, inexpensive, efficient use of\nhardware, provide protection, unifying abstraction (files, byte\nstreams)\u003c/li\u003e\n\u003cli\u003eFile System:\n\u003cul\u003e\n\u003cli\u003eData model: Before Unix (imposed structure), Unix (uninterpreted\nbytes, variable-length)\u003c/li\u003e\n\u003cli\u003eAPI: Before Unix (no buffer cache, separate \u0026amp; async APIs), Unix\n(buffer cache, simple APIs e.g. \u003ccode\u003ewrite(fd, buffer, length)\u003c/code\u003e)\u003c/li\u003e\n\u003cli\u003eFile names \u0026amp; dirs: hierarchical, mount, dirs stored like files\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eDevices: device independent I/O, streaming interface, driver, ioctl\nfor corner cases\u003c/li\u003e\n\u003cli\u003eProtection Model: 7-bit R/W/X, user/other; \u003ccode\u003eset-user-id\u003c/code\u003e → execute\nw/ owner\u0026rsquo;s permissions, enables protected subsystems\u003c/li\u003e\n\u003cli\u003eProcesses: \u003ccode\u003efork()\u003c/code\u003e \u0026amp; \u003ccode\u003eexec()\u003c/code\u003e vs \u003ccode\u003ecreateProcess()\u003c/code\u003e. Pros \u003ccode\u003efork()\u003c/code\u003e:\neasy concurrency, \u003ccode\u003efork\u003c/code\u003e+\u003ccode\u003eexec\u003c/code\u003e combo, simple; Pros\n\u003ccode\u003ecreateProcess()\u003c/code\u003e: flexible, efficient; Cons \u003ccode\u003efork()\u003c/code\u003e:\nimplementation complexity, not thread safe, insecure\u003c/li\u003e\n\u003cli\u003eSummary: unifying abstractions (files), careful decomposition\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"singularity\"\u003e\n \u003cspan\u003eSingularity\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#singularity\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eMotivation: software-based isolation, reconsider OS design, security\nvulnerabilities, failures, robustness\u003c/li\u003e\n\u003cli\u003eSingularity Approach: language features (e.g. bounds checking),\nverification, apply to an OS\u003c/li\u003e\n\u003cli\u003eDesign of Singularity: SIPs (domains communicate with messages), one\naddress space, CBCs, Manifest-Based Program (MBP)\u003c/li\u003e\n\u003cli\u003eCBC: Contract-based channels; Client ← channel → sever; state\nmachine\u003c/li\u003e\n\u003cli\u003eSummary: Software-based isolation\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch1 id=\"synchronization\"\u003e\n \u003cspan\u003eSynchronization\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#synchronization\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h1\u003e\n\u003ch2 id=\"monitors\"\u003e\n \u003cspan\u003eMonitors\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#monitors\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eMonitor: Sync by compiler, enforced at runtime. Guarantees mutual\nexclusion; one thread at a time. Java: \u003ccode\u003esynchronize\u003c/code\u003e keyword.\nMonitor invariant: safety property; holds on thread entry/exit.\nCondition Variable: for thread progress (liveness) in monitor\u003c/li\u003e\n\u003cli\u003eHoare semantics: \u003ccode\u003ewait\u003c/code\u003e: block \u0026amp; release lock; \u003ccode\u003esignal\u003c/code\u003e: wake up\nwaiter, context switch; Invariant \u003ccode\u003eI\u003c/code\u003e: \u003ccode\u003eI wait I \u0026amp; B\u003c/code\u003e,\n\u003ccode\u003eB \u0026amp; I signal I\u003c/code\u003e; Condition \u003ccode\u003eB\u003c/code\u003e: situation that needs wait\u003c/li\u003e\n\u003cli\u003eMesa semantics: \u003ccode\u003ewait\u003c/code\u003e: same as Hoare; \u003ccode\u003esignal/notify\u003c/code\u003e:\n\u003ccode\u003ewhile (!condition) wait()\u003c/code\u003e; \u003ccode\u003eI wait I\u003c/code\u003e, \u003ccode\u003exB xI signal xI xB\u003c/code\u003e\u003c/li\u003e\n\u003cli\u003eTrends: Hoare → compiler; Java: compiler handles locks, explicit CV;\nC++/Rust: hybrid\u003c/li\u003e\n\u003cli\u003eSummary: language support for sync monitors; Hoare vs Mesa\nsemantics; challenges (aborts, nesting)\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch1 id=\"scalability\"\u003e\n \u003cspan\u003eScalability\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#scalability\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h1\u003e\n\u003ch2 id=\"rcu\"\u003e\n \u003cspan\u003eRCU\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#rcu\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eMulticore CPUs ( 2000 first CPUs with 2 cores). Scalable: perf\nincreases linearly w/ # cores\u003c/li\u003e\n\u003cli\u003eGoals: Concurrent reads (even during updates); Low overhead;\nDeterministic completion time\u003c/li\u003e\n\u003cli\u003eApproaches: (1) spin lock (2) read-write lock: multi readers OR 1\nwriter. Drawbacks: instructions, readers wait for writer, space \u0026amp;\nexecution overhead\u003c/li\u003e\n\u003cli\u003eRCU ideas: (1) writes make a copy (2) Memory barriers to enforce\nordering (problem: use-after-free) (3) grace period: writers wait\nuntil all CPUs context switch; readers can\u0026rsquo;t hold pointer through\ncontext switch\u003c/li\u003e\n\u003cli\u003eLinux\u0026rsquo;s RCU API:\n\u003cul\u003e\n\u003cli\u003ereaders: rcu_read_lock (disable preemption), rcu_read_unlock\n(enable preemption), rcu_dereference(pointer)\u003c/li\u003e\n\u003cli\u003ewriter: synchronize_rcu (wait for all cores to context switch),\ncall_rcu(call_back, argument) (async),\nrcu_assign_pointer(pointer_addr, pointer)\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eRCU vs read-write locks:\n\u003cul\u003e\n\u003cli\u003eReader: RCU imposes almost no cost, allows reads while writes\noccur\u003c/li\u003e\n\u003cli\u003eWriter: RCU can take longer, better with fewer writes (not\nlatency-sensitive)\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eDrawbacks: complex, space overheads from copy, hard to use outside\nkernel, confusing semantics, relies on frequent context switches,\nhard to apply to some structures, reads ≫ writes\u003c/li\u003e\n\u003cli\u003eSummary: multiple CPU cores → scalable synchronization; no waiting\nfor readers, grace period for writers\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"scalable-commutativity-rule\"\u003e\n \u003cspan\u003eScalable Commutativity Rule\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#scalable-commutativity-rule\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003e\u003cstrong\u003eScalable commutativity rule\u003c/strong\u003e: if interface ops commute → exists\nconflict-free (scalable) implementation\u003c/li\u003e\n\u003cli\u003eMemory accesses: read-only or write on 1 core: scalable; write a\ncache line read/written by another core: not scalable\u003c/li\u003e\n\u003cli\u003eCommutativity: getpid() always commutes; open(\u0026rsquo;/a/new.txt\u0026rsquo;,\nO_CREATE) \u0026amp; open(\u0026rsquo;/b/new.txt\u0026rsquo;, O_CREATE) commute\u003c/li\u003e\n\u003cli\u003eSIM-commutativity: State-dependent, Interface-based, Monotonic (any\nprefix should be commutative)\u003c/li\u003e\n\u003cli\u003eCommutative interfaces: POSIX; decompose complex ops (fork() →\nposix_spawn, CreateProcess); allow non-determinism (file\ndescriptors)\u003c/li\u003e\n\u003cli\u003eCaveats: failing rule ≠ not scalable; all ops passing ≠ scalable OS\n(implementation could conflict)\u003c/li\u003e\n\u003cli\u003eSummary: Scalability at interface level, ops commute → scalable\nimplementation exists\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch1 id=\"virtual-memory\"\u003e\n \u003cspan\u003eVirtual Memory\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#virtual-memory\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h1\u003e\n\u003ch2 id=\"virtual-memory-management-in-the-vaxvms-operating-system\"\u003e\n \u003cspan\u003eVirtual Memory Management in the VAXVMS Operating System\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#virtual-memory-management-in-the-vaxvms-operating-system\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eGoals: support diff hardware \u0026amp; apps; efficiency (limited mem, slow\nI/O)\u003c/li\u003e\n\u003cli\u003eAddress space: virtual → physical addresses\n\u003cul\u003e\n\u003cli\u003eHW support: base+length registers (exceed→segfault,\nupdate@context switch, security issues like spectre→mitigation:\nKPTI)\u003c/li\u003e\n\u003cli\u003eMultiple page tables for P1 \u0026amp; P0 to save space\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eSpace efficiency: 32-bit addr space; small 512B pages; allow page\ntable swap out\u003c/li\u003e\n\u003cli\u003eOS page table: in phys mem to avoid faulting; modern OS: pin root\npage table only\u003c/li\u003e\n\u003cli\u003eTLB: cache of PTEs to avoid 2 mem accesses per access; split b/w\nuser \u0026amp; kernel\u003c/li\u003e\n\u003cli\u003eLocal page replacement: simple FIFO (today: LRU variants)\u003c/li\u003e\n\u003cli\u003ePage caching: 2nd chance; proactively reclaim pages→free list;\nmaintains modified list\u003c/li\u003e\n\u003cli\u003eClustering: bring/write multiple virtually contiguous 512B pages in\nphys contig mem\u003c/li\u003e\n\u003cli\u003eSummary: address space; HW support for mem mgmt\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"machine-independent-virtual-memory-management-for-paged-uniprocessor-and-multiprocessor-architectures\"\u003e\n \u003cspan\u003eMachine-Independent Virtual Memory Management for Paged Uniprocessor and Multiprocessor Architectures\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#machine-independent-virtual-memory-management-for-paged-uniprocessor-and-multiprocessor-architectures\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eVirtual memory for Mach (Richard Rashid → founded MSR, Avadis\nTevanian → Apple CTO)\u003c/li\u003e\n\u003cli\u003eIBM RTPC (RISC PC) → PowerPC\u003c/li\u003e\n\u003cli\u003eGoals: machine independence, evaluate hardware platforms, support\nsparse address spaces\u003c/li\u003e\n\u003cli\u003eResident page table: inverted page table, physical pages mapping to\naddress space items\u003c/li\u003e\n\u003cli\u003eMemory object: backing store (executable, disk file, paging file,\netc.), \u003ccode\u003euserfaultfd\u003c/code\u003e for user-controlled swapping\u003c/li\u003e\n\u003cli\u003epmap: hardware-dependent VM data \u0026amp; code (page frames, address bits,\nprotection, paging vs segmentation, page table structure)\u003c/li\u003e\n\u003cli\u003eSummary: machine independence of VM\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch1 id=\"distributed-oses\"\u003e\n \u003cspan\u003eDistributed OSes\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#distributed-oses\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h1\u003e\n\u003ch2 id=\"the-distributed-v-kernel-and-its-performance-for-diskless-workstations\"\u003e\n \u003cspan\u003eThe Distributed V Kernel and its Performance for Diskless Workstations\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#the-distributed-v-kernel-and-its-performance-for-diskless-workstations\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eDiskless workstation claims: diskless = local disk; manageability;\nmessage IPC enables high perf file I/O\u003c/li\u003e\n\u003cli\u003eV IPC: inherit from Thoth, extend to distributed networks;\nsynchronous; small fixed-size messages (32 bytes, send/reply);\nseparate data transfer mechanism (\u003ccode\u003emoveTo/moveFrom\u003c/code\u003e,\n\u003ccode\u003ereceiveWithSegment/replyWithSegment\u003c/code\u003e); perf optimizations\n(rawEthernet)\u003c/li\u003e\n\u003cli\u003eNetwork Penalty: minimum transfer time\u003c/li\u003e\n\u003cli\u003eKernel Primitives: same machine vs over network; clock speed\nmatters; remote adds ms of overhead (benefits of offloading, file\naccess takes 20ms)\u003c/li\u003e\n\u003cli\u003eFile access: streaming (network, disk IO no prefetching); I/O bound\n→ blocked on disk no matter what (needs strong assumption that\nsequential prefetching doesn\u0026rsquo;t change perf); not I/O bound → doesn\u0026rsquo;t\nmatter\u003c/li\u003e\n\u003cli\u003eBenchmark from micro to larger, end-to-end ones\u003c/li\u003e\n\u003cli\u003eSummary: diskless workstations; arguments based on performance\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"sprite\"\u003e\n \u003cspan\u003eSprite\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#sprite\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eLarger memories → caching\u003c/li\u003e\n\u003cli\u003eNetworking: connected workstations; management/administration;\ntransparency; sharing (single global namespace, potentially idle\nmachines → process migration (today: migration done on container/VM\nlevel), home machine (if incompatible b/c of different kernel env,\nRPC back to home machine))\u003c/li\u003e\n\u003cli\u003eMultiprocessor: sharing \u0026amp; parallelism → shared fork\u003c/li\u003e\n\u003cli\u003eOS adaptation to multicore: monitor at fine-grained level\u003c/li\u003e\n\u003cli\u003eCaching: sharing is rare; sequential (use version #); concurrent\n(disable caching on file-by-file basis); strategy: optimize for\ncommon case\u003c/li\u003e\n\u003cli\u003eSummary: file caching protocol; technology trends\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"plan-9\"\u003e\n \u003cspan\u003ePlan-9\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#plan-9\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eUNIX not a good fit: workstations instead of mainframes\n(interconnected vs connecting to central mainframe); distributed\nnature; admin/management; cost; graphics, networking\u003c/li\u003e\n\u003cli\u003ePlan 9 Approach: Terminal, CPU, FS, connected by network\u003c/li\u003e\n\u003cli\u003eBenefit: cost; longevity (can replace some components only)\u003c/li\u003e\n\u003cli\u003eUnifying abstraction \u0026ndash; files: everything is a file (\u003ccode\u003e/dev/mouse\u003c/code\u003e\ncan be local or remote in distributed setting, \u003ccode\u003e/proc\u003c/code\u003e clean way to\nexamine \u0026amp; control running processes, \u003ccode\u003e/net\u003c/code\u003e, \u003ccode\u003e/dev/bitblt\u003c/code\u003e,\n\u003ccode\u003e/dev/con\u003c/code\u003e)\u003c/li\u003e\n\u003cli\u003eNamespaces: local namespaces; easy to customize; implement: \u003ccode\u003emount\u003c/code\u003e,\n\u003ccode\u003eumount\u003c/code\u003e, \u003ccode\u003ebind\u003c/code\u003e; union directory (e.g., search single \u003ccode\u003e/bin\u003c/code\u003e which\nall \u003ccode\u003e/bin\u003c/code\u003es mount to)\u003c/li\u003e\n\u003cli\u003e9P: network protocol, unifying all file operations (NFS, SSH, X11);\napplications responsible for interpreting semantics; similar to HTTP\nwhere applications interpret bytes\u003c/li\u003e\n\u003cli\u003eStorage hierarchy: memory - cache; disk - file server; WORM (Write\nOnce Read Many) - drawback: space, scalability, unreliability, slow\nto repopulate\u003c/li\u003e\n\u003cli\u003eProcess: \u003ccode\u003erfork\u003c/code\u003e - fine-grained sharing (similar to \u003ccode\u003eclone\u003c/code\u003e in\nLinux); concurrency-oriented language Alef influenced Go\u003c/li\u003e\n\u003cli\u003eImpact: cost of disk not really important; privacy\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"legoos\"\u003e\n \u003cspan\u003eLegoOS\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#legoos\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eBuilding OS for disaggregated hardware\u003c/li\u003e\n\u003cli\u003eMonolithic server vs Hardware disaggregation\u003c/li\u003e\n\u003cli\u003eBenefits: fine-grained failure domains; scalability (independently);\nresource efficiency (solves bin-packing problem); adding specialized\nhardware; non-goal: performance; taking early ideas to the extreme →\ndisaggregate everything\u003c/li\u003e\n\u003cli\u003eLegoOS: Splitkernel (monitors: pComp, mComp, sComp)\u003c/li\u003e\n\u003cli\u003eEvaluation: small working sets (performance not that bad); large\nworking sets (actually better); resource utilization?\u003c/li\u003e\n\u003cli\u003eLimitations: assume no sharing of memory; security; policy\u003c/li\u003e\n\u003cli\u003eDistributed OS adopted? Disaggregated storage, but nothing more\naggressive\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"the-performance-of-μ-kernel-based-systems\"\u003e\n \u003cspan\u003eThe Performance of μ-Kernel-Based Systems\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#the-performance-of-%ce%bc-kernel-based-systems\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eGoals: Microkernels performance - fundamental? \u003cstrong\u003e5-10%\u003c/strong\u003e;\nExtensibility, Specialization (extensibility \u0026ndash; new features\n(cache), specialization \u0026ndash; existing services (RPC)); Do we need\ncolocation? \u003cstrong\u003eno\u003c/strong\u003e\u003c/li\u003e\n\u003cli\u003eAbstractions: threads; IPC (messages); address spaces\u003c/li\u003e\n\u003cli\u003eL4 architecture:\u003c/li\u003e\n\u003cli\u003eSyscall: Standard: \u003ccode\u003eread()\u003c/code\u003e → libc → syscall; L4: call modified libc\nto send message to L4, L4 decides to route to Linux process (more\noverhead b/c more boundary crossings)\u003c/li\u003e\n\u003cli\u003ePage faults → similar; Linux → two parts arch-dependent vs not;\nmodified arch-dependent part to use L4 as a platform, other part\nunmodified\u003c/li\u003e\n\u003cli\u003eMemory management: hierarchical address spaces; page tables\u003c/li\u003e\n\u003cli\u003eSummary: design of modern microkernel; performance = monolithic OS;\nspecialization, extension\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch1 id=\"extending-the-os\"\u003e\n \u003cspan\u003eExtending the OS\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#extending-the-os\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h1\u003e\n\u003ch2 id=\"exokernel\"\u003e\n \u003cspan\u003eExokernel\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#exokernel\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eGoals: expose hardware; push everything → user level\u003c/li\u003e\n\u003cli\u003eBenefits: specialization; extensibility; lower overhead\u003c/li\u003e\n\u003cli\u003eTrack resource ownerships → tables\u003c/li\u003e\n\u003cli\u003eProtection via secure bindings (decouple authorization from access)\u003c/li\u003e\n\u003cli\u003eResource revocation\u003c/li\u003e\n\u003cli\u003eMemory management: TLB miss → trap to exokernel (downside: overhead;\nsolution: software TLB)\u003c/li\u003e\n\u003cli\u003eSummary: move OS to user-level; untrusted library OS - manage HW\nresource directly?\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch1 id=\"virtualization\"\u003e\n \u003cspan\u003eVirtualization\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#virtualization\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h1\u003e\n\u003ch2 id=\"the-origin-of-the-vm370-time-sharing-system\"\u003e\n \u003cspan\u003eThe Origin of the VM/370 Time-Sharing System\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#the-origin-of-the-vm370-time-sharing-system\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eWhy VM? Continuous operation; isolation; full access to hardware; OS\ndevelopment; emulate new hardware\u003c/li\u003e\n\u003cli\u003eTrap and emulate: trap to hypervisor which emulates the behavior\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"xen\"\u003e\n \u003cspan\u003eXen\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#xen\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eAWS built upon Xen; hence Xen became popular\u003c/li\u003e\n\u003cli\u003eWhy VMs? Data centers/cloud computing\u003c/li\u003e\n\u003cli\u003eRequirements: isolation (security \u0026amp; performance); heterogeneous\nOSes; unmodified applications; performance; scalability; accounting\u003c/li\u003e\n\u003cli\u003eChallenge: virtualizing x86 (TLB - hardware managed (x86 ARM);\ndoesn\u0026rsquo;t trap on every privileged instruction; different side effects\n\u0026ndash; popf)\u003c/li\u003e\n\u003cli\u003eCPU: Paravirtualization (modify guest OS); privileged instructions →\nhyper call; VMWare → binary rewriting (scan next basic block \u0026amp; see\nif there are any instructions, call into hypervisor); syscall\noptimization (insert into hypervisor)\u003c/li\u003e\n\u003cli\u003eXen \u0026ndash; similar to Exokernel: Write → trap to hypervisor; Guest OS\ncan still walk page table as normal x86\u003c/li\u003e\n\u003cli\u003eVMWare approach: shadow page table; use binary rewriting to copy to\nshadow page table\u003c/li\u003e\n\u003cli\u003eI/O: disk, network, mouse; too many device drivers\u003c/li\u003e\n\u003cli\u003eHardware support for Virtualization\u003c/li\u003e\n\u003cli\u003eSummary: trap \u0026amp; emulate; x86 hard to virtualize; paravirtualization;\nbinary rewriting; hardware support\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"container-based-operating-system-virtualization-vserver\"\u003e\n \u003cspan\u003eContainer-Based Operating System Virtualization (VServer)\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#container-based-operating-system-virtualization-vserver\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eBenefits vs VMs: lower overhead, more efficiency \u0026amp; scalability\n(overcommit)\u003c/li\u003e\n\u003cli\u003eDrawbacks vs VMs: less isolation, same OS, need to modify OS\u003c/li\u003e\n\u003cli\u003eIsolation: fault (buggy, malicious), resource (CPU, memory),\nsecurity (access to info)\u003c/li\u003e\n\u003cli\u003eContexts: Namespaces (Process PIDs, Users UIDs/GIDs, Networks IP\nports, Mount files), Filters (local objects only)\u003c/li\u003e\n\u003cli\u003eResource Allocations:\n\u003cul\u003e\n\u003cli\u003eCPU: Token Bucket for CPI scheduling \u0026amp; IO\u003c/li\u003e\n\u003cli\u003eMemory, Network, Disk\u003c/li\u003e\n\u003cli\u003eFile System: chroot (part of FS container can see), CoW for\nsharing common files (1 container 500MB, 10 containers 700MB)\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eEmulation: Xen higher overhead for microbenchmarks, CPU \u0026amp; memory\nintensive similar to Xen\u003c/li\u003e\n\u003cli\u003eSummary: Containers virtualize at OS level; implemented w/\nnamespaces \u0026amp; filters\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"firecracker\"\u003e\n \u003cspan\u003eFirecracker\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#firecracker\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eServerless apps: goals (performance \u0026amp; utilization), simple\nfunction/script, storage usually elsewhere (e.g., s3)\u003c/li\u003e\n\u003cli\u003eUse cases: data streaming, web apps, edge settings (IoT),\nevent-driven/on-demand, short-lived (≤15 min)\u003c/li\u003e\n\u003cli\u003eBenefits: auto-scaling (0→∞), manageability, pay-per-use, fault\ntolerance, simplicity\u003c/li\u003e\n\u003cli\u003eMotivation requirements:\n\u003cul\u003e\n\u003cli\u003eFast switching ( 100ms) for lower latency \u0026amp; charging\u003c/li\u003e\n\u003cli\u003eDensity (1000s of functions on single machine)\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eBefore Firecracker: language runtime in containers\u003c/li\u003e\n\u003cli\u003eFirecracker: hardware-based virtualization per function to minimize\noverhead\u003c/li\u003e\n\u003cli\u003eEvaluation:\n\u003cul\u003e\n\u003cli\u003eBoot-time 125ms (remove devices\u0026hellip;)\u003c/li\u003e\n\u003cli\u003eMemory 3MB per VMM\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eSummary: Hardware-based virtualization \u0026amp; optimization for practical\nrequirements (main benefits, nothing else new)\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch1 id=\"scheduling\"\u003e\n \u003cspan\u003eScheduling\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#scheduling\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h1\u003e\n\u003ch2 id=\"the-linux-scheduler-a-decade-of-wasted-cores\"\u003e\n \u003cspan\u003eThe Linux Scheduler, A Decade of Wasted Cores\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#the-linux-scheduler-a-decade-of-wasted-cores\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eGoals: Work conserving (no idle core if runnable threads), efficient\n(cache behavior), policies (fairness, prioritization), power,\ngeneral purpose\u003c/li\u003e\n\u003cli\u003eSingle-core scheduling: Weighted (vruntime), preemptedl Red-black\ntree (pick lowest vruntime \u0026amp; run next)\u003c/li\u003e\n\u003cli\u003eMulti-core:\n\u003cul\u003e\n\u003cli\u003eAvoid synchronized access\u003c/li\u003e\n\u003cli\u003eChallenges: load imbalance, complex cache hierarchies\u003c/li\u003e\n\u003cli\u003ePeriodic → load factor\u003c/li\u003e\n\u003cli\u003e\u0026ldquo;Emergency\u0026rdquo; (idle core)\u003c/li\u003e\n\u003cli\u003eThreads created/wakeup\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eMetrics: # of threads, sum of weights, load factor (weights \u0026amp; CPU\nutilization)\u003c/li\u003e\n\u003cli\u003eAlgorithm: hierarchy approach\u003c/li\u003e\n\u003cli\u003eBugs:\n\u003cul\u003e\n\u003cli\u003eGroup imbalance bug (fix: replace average w/ minimum)\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eSummary: Scalable schedule (multiple runqueues), load imbalance, CPU\nscheduling is tricky\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"arachne\"\u003e\n \u003cspan\u003eArachne\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#arachne\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eMotivating apps for Arachne: low-latency key-value stores, storage\n(RAMCloud), web services (care about p99 tail latency)\u003c/li\u003e\n\u003cli\u003eUser-level threads: less overhead, kernel-level events hidden,\ncustomization, portable\u003c/li\u003e\n\u003cli\u003eKernel threads: hardware parallelism, OS awareness, better isolation\u003c/li\u003e\n\u003cli\u003eArachne: all user-level threading, N → constantly changing, 2-level\nscheduling\u003c/li\u003e\n\u003cli\u003eThreading library:\n\u003cul\u003e\n\u003cli\u003ecooperative (no preemption), call \u003ccode\u003eyield()\u003c/code\u003e, blocking kernel\ncalls (async I/O, rare page faults)\u003c/li\u003e\n\u003cli\u003eload balancing: at thread creation, power-of-2 choices\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eCore Arbiter: ask then revoke, blocking read\u003c/li\u003e\n\u003cli\u003eSummary: 2-level scheduling (user \u0026amp; kernel), microseconds-scale app\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"implementing-remote-procedure-calls\"\u003e\n \u003cspan\u003eImplementing Remote Procedure Calls\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#implementing-remote-procedure-calls\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eGoals of Remote Procedure Call (RPC): efficiency, security,\nidentical semantics as local procedure calls (e.g., timeouts),\ngenerality\u003c/li\u003e\n\u003cli\u003eOptimizations:\n\u003cul\u003e\n\u003cli\u003eTransport: piggy-backing ACKs with data → synchronous (one\nthread, one outstanding RPC)\u003c/li\u003e\n\u003cli\u003eProcesses: pool of idle processes, process hints\u003c/li\u003e\n\u003cli\u003eConnections: no setup/teardown, few states for inactive RPC\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eBinding: at runtime (Grapevine)\u003c/li\u003e\n\u003cli\u003eException \u0026amp; errors: shutdown? retry?\u003c/li\u003e\n\u003cli\u003eRPC today: datacenters (Thrift, gRPC), browser\u003c/li\u003e\n\u003cli\u003eSummary: make remote functionality appear local, optimize design for\nRPC\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch1 id=\"networking\"\u003e\n \u003cspan\u003eNetworking\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#networking\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h1\u003e\n\u003ch2 id=\"eliminating-receive-livelock-in-an-interrupt-driven-kernel\"\u003e\n \u003cspan\u003eEliminating Receive Livelock in an Interrupt-Driven Kernel\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#eliminating-receive-livelock-in-an-interrupt-driven-kernel\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eApplications: routing (firewall, NAT, VPN, load balancer),\nmonitoring (passive), services (file service, web server). High\ntraffic volumes, not rate controlled\u003c/li\u003e\n\u003cli\u003eNetwork Stack - receive many queues: bursts, batching at different\nlayers, organizational\u003c/li\u003e\n\u003cli\u003eProblem: receive livelock → overload, spend all time on interrupt\nprocessing.\u003c/li\u003e\n\u003cli\u003ePolling: check for incoming packets. Polling cons: waste CPU cycles\nat low load, pros: efficient at high load. Interrupt cons: high\noverhead at high load, pros: lower latency (high priority),\nproportional work\u003c/li\u003e\n\u003cli\u003eSolutions: hybrid approach (interrupt \u0026amp; polling), round robin,\nremove layer of queueing\u003c/li\u003e\n\u003cli\u003eSummary: hybrid approach (polling + interrupts), on overloads → drop\ntasks as early as possible\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"demikernel\"\u003e\n \u003cspan\u003eDemikernel\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#demikernel\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003ePortability: heterogeneous hardware (DPDK, RDMA), uniform interface\nfor applications\u003c/li\u003e\n\u003cli\u003ens-scale overhead, simplifying development\u003c/li\u003e\n\u003cli\u003eLibrary OS: optimized libOSes\u003c/li\u003e\n\u003cli\u003eGeneral API: POSIX variant (queues instead of files, async\ninterfaces)\u003c/li\u003e\n\u003cli\u003eMemory Management: zero copy, DMA capable memory (pinned,\nregistered), use-after-free (reference counts). No concerns for\nregular OS b/c copy buffers\u003c/li\u003e\n\u003cli\u003eCPU management: coroutines (similar to user-level threads \u0026amp;\nArachne), 12 cycles to switch, non-preemptive\u003c/li\u003e\n\u003cli\u003eSummary: General purpose API for Kernel-bypass I/O. Techniques:\nzero-copy I/O, memory management, scheduling\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"snap\"\u003e\n \u003cspan\u003eSnap\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#snap\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eGoals: easy deployment, efficient CPU scheduling, high-performance\nnetworking, easy development \u0026amp; optimization\u003c/li\u003e\n\u003cli\u003eLibOS (Demikernel) vs Microkernel (Snap):\n\u003cul\u003e\n\u003cli\u003eLibOS: lower latency, better isolation, more tailored\u003c/li\u003e\n\u003cli\u003eMicrokernel: centralized management, easy deployment, can have\nhigher overhead, handles scheduling of network processing\n(polling), explicit control\u003c/li\u003e\n\u003cli\u003eBoth: userspace, decoupled with OS\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eThreading: kernel-level threads\u003c/li\u003e\n\u003cli\u003eIPC: multicore to optimize\u003c/li\u003e\n\u003cli\u003eSnap Microkernel:\n\u003cul\u003e\n\u003cli\u003econtrol \u0026amp; data plane\u003c/li\u003e\n\u003cli\u003eengines: dedicated cores, spreading engines (scales CPU with\nload), compacting engines (fewer cores, higher tail latency)\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eUpgrades:\n\u003cul\u003e\n\u003cli\u003estart new Snap, for each engine: write state to memory, read\nstate in new Snap, **brownout, pause old snap, copy changes,\nstart new engine, **blackout, 250ms median\u003c/li\u003e\n\u003cli\u003ekill old snap\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eSummary: microkernel-based approach (easier upgrades), scheduling\nmodes for engines\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch1 id=\"io-and-file-systems\"\u003e\n \u003cspan\u003eI/O and File Systems\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#io-and-file-systems\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h1\u003e\n\u003ch2 id=\"fast-file-system\"\u003e\n \u003cspan\u003eFast-File System\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#fast-file-system\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eUnix Fast File System (FFS)\u003c/li\u003e\n\u003cli\u003eDisk: Cylinder Group (read together relatively closely), Access Time\n(seek ms, rotation 3600-15K, transfer - read sector), Disk\nInterface (FFS - specify platter, surface, etc; Block interface -\nmodern)\u003c/li\u003e\n\u003cli\u003eUnix File System: Disk blocks 4KB. Optimize for common case of small\nfiles\u003c/li\u003e\n\u003cli\u003eFFS Objectives: Performance, Reliability\u003c/li\u003e\n\u003cli\u003eLimitations: Layout (lots of seeks between inode and data, block\nsize (512-byte blocks → 1024 bytes)\u003c/li\u003e\n\u003cli\u003eFFS Solutions:\n\u003cul\u003e\n\u003cli\u003erandom block placement → cylinder group (related blocks\ntogether, same file has blocks in same group)\u003c/li\u003e\n\u003cli\u003einodes far from data blocks → cylinder groups\u003c/li\u003e\n\u003cli\u003elow bandwidth utilization → increase block size to 4KB\u003c/li\u003e\n\u003cli\u003ewaste with large block sizes (internal fragmentation) →\nfragments\u003c/li\u003e\n\u003cli\u003esmall max file size → increase block size to 4KB (indirect\nblocks also bigger)\u003c/li\u003e\n\u003cli\u003erecovery → replicated to multiple known locations\u003c/li\u003e\n\u003cli\u003edevice oblivious → parameterized file system to the device\n(place blocks in rotationally optimal ways; not used today by\nDRAM on disk)\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eAllocation: global (which cylinder groups), local (which blocks in a\ncylinder? leave 10% unused)\u003c/li\u003e\n\u003cli\u003eSummary: aware of device characteristics\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"log-structured-file-system\"\u003e\n \u003cspan\u003eLog-Structured File System\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#log-structured-file-system\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eTrends: large memory (larger file buffer cache), faster CPUs,\nwrite-heavy disk workloads, disk (increased capacity, seek time\nstill slow, increase transfer bandwidth), small files\u003c/li\u003e\n\u003cli\u003eProblems with FFS: layout (seek back and forth still a problem),\nsynchronous writes for metadata\u003c/li\u003e\n\u003cli\u003eLFS Approach: think of disk as log (1MB segment)\u003c/li\u003e\n\u003cli\u003eChallenge: finding data inodes (inode map), free space (garbage\ncollection)\u003c/li\u003e\n\u003cli\u003eCleaning:\n\u003cul\u003e\n\u003cli\u003ecopy in-use blocks to log (cost: wasted bandwidth)\u003c/li\u003e\n\u003cli\u003epolicy questions: cold (clean more aggressively) and hot\nsegments (wait longer), cost-benefit policy\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eUsed in SSD (disks wear out if written repeatedly), Flash\nTranslation Layer (FTL)\u003c/li\u003e\n\u003cli\u003eSummary: optimize for trends, writing data sequentially in a log\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"soft-updates\"\u003e\n \u003cspan\u003eSoft Updates\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#soft-updates\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eMetadata: inode, directories, bitmaps (need to always be consistent)\u003c/li\u003e\n\u003cli\u003eOrdering constraints:\n\u003cul\u003e\n\u003cli\u003eno dangling pointers\u003c/li\u003e\n\u003cli\u003enever reset old ptr until new ptr persists (e.g., rename)\u003c/li\u003e\n\u003cli\u003eno reuse until old pointers are removed\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eSoft updates:\n\u003cul\u003e\n\u003cli\u003etrack dependencies, write in a safe order (asynchronous, less\ndisk write, consistency)\u003c/li\u003e\n\u003cli\u003ecircular dependencies solution: fine-grained dependency\ntracking, undo/redo\n\u003cul\u003e\n\u003cli\u003ememory overhead (keep track of versions), cpu overhead\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eOptimization: write many blocks at once, temporary files (don\u0026rsquo;t need\nto write)\u003c/li\u003e\n\u003cli\u003eEvaluation: Compare FFS \u0026ldquo;Conventional\u0026rdquo;, FFS async writes \u0026ldquo;No Order\u0026rdquo;\n( 5% differences), Soft Updates, Write-ahead logging. Don\u0026rsquo;t need\nrecovery, Con: add complexity\u003c/li\u003e\n\u003cli\u003eSummary: metadata consistency \u0026amp; ordering constraints, soft updates\n(fine-grained dependency tracking, rollback)\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"splitfs\"\u003e\n \u003cspan\u003eSplitFS\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#splitfs\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eSplitFS: hybrid approach \u0026ndash; data (user space), metadata (kernel)\u003c/li\u003e\n\u003cli\u003eAppends:\n\u003cul\u003e\n\u003cli\u003eneeds to update metadata\u003c/li\u003e\n\u003cli\u003estaging file → relink\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eSummary: hybrid design, staging file \u0026amp; relink operation\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"the-google-file-system\"\u003e\n \u003cspan\u003eThe Google File System\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#the-google-file-system\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eGFS Setting: scale, failure is common (scale \u0026amp; cheap machines,\nautomated recovery), write (sequential \u0026amp; append), performance \u0026gt;\nconsistency, throughput \u0026gt;\u0026gt; latency, large files (100 MB - 100 GB),\nreads (sequential and random), concurrent writes (MapReduce)\u003c/li\u003e\n\u003cli\u003eGFS: user-level file system, no POSIX interface\u003c/li\u003e\n\u003cli\u003eFiles: chunks (64 MB), stored as Linux files\u003c/li\u003e\n\u003cli\u003eNamespaces: hierarchical (\u003ccode\u003ea/b/foo.txt\u003c/code\u003e)\u003c/li\u003e\n\u003cli\u003eDirectories: illusion, file path → files\u003c/li\u003e\n\u003cli\u003eGFS Architecture: Master, Chunkserver, Clients\u003c/li\u003e\n\u003cli\u003eRead: version number, no explicit caching\u003c/li\u003e\n\u003cli\u003eWrite: pick primary \u0026amp; 3 chunk servers, let client know where to\nwrite; primary writes to next chunk server, and so on (full network\nbandwidth); chunk servers rely, primary says commit when all done\u003c/li\u003e\n\u003cli\u003eConcurrency appends: record append, \u003ccode\u003ewrite()\u003c/code\u003e (specific offset) →\nprimary orders record append, at-least-once semantics, inconsistency\nfor apps (duplicate records (record ID), padding \u0026amp; fragments\n(checksums)), assumed GFS only used internally by experts\u003c/li\u003e\n\u003cli\u003eSummary: first large-scale distributed file system, challenges\n(scale, failures, consistency)\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"push-button-verification-of-file-systems-via-crash-refinement\"\u003e\n \u003cspan\u003ePush-Button Verification of File Systems via Crash Refinement\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#push-button-verification-of-file-systems-via-crash-refinement\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eGoals: low effort, deterministic, insightful (counter examples),\nhigh performance\u003c/li\u003e\n\u003cli\u003eUser\u0026rsquo;s Perspective: Specification ((dir inode #, name) → child inode\n#), Implementation, Consistency Invariant\u003c/li\u003e\n\u003cli\u003eAutomated Verification:\n\u003cul\u003e\n\u003cli\u003ecrash-free, crashes (non-determination)\u003c/li\u003e\n\u003cli\u003ecrash refinement\u003c/li\u003e\n\u003cli\u003erecovery function\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eTools for verification: symbolic execution → SMT encoding → SMT\nsolver (e.g., Z3)\u003c/li\u003e\n\u003cli\u003eSummary: correctness for implementation, notion of crash refinement\u003c/li\u003e\n\u003c/ul\u003e","date_modified":"2026-06-10T00:40:55-07:00","date_published":"2024-04-28T21:12:08+08:00","id":"https://www.xiyan.dev/writings/2024/04/28/operating-systems-papers-readings-note/","summary":"Reading notes for Graduate Operating Systems at UCSD. The notes are based on the readings and discussions from the course.\n","tags":["Computer Science","Paper Reading","CSE221"],"title":"Operating Systems Papers Readings Note","url":"https://www.xiyan.dev/writings/2024/04/28/operating-systems-papers-readings-note/"},{"content_html":"\u003cp\u003eBoosting Large Language Models (LLMs) applications with tools to interact with third-party services enables LLMs to retrieve updated knowledge and perform actions on behalf of users. However, the added capability brings security and privacy risks. In the current paradigm, users delegate potentially sensitive resources to LLM Apps, which makes the platforms overprivileged. For instance, malicious platforms or rogue models can exploit shared email-sending or TAP platform tokens stealthily. We propose LLMacaroon, a practical and secure architecture that distrusts applications for sharing sensitive resources and shifts control back to users. LLMacaroon achieves flexible, controlled sharing via macaroons and improves transparency and control via a local action proxy with optionally human in the loop. We demonstrate that LLMacaroon requires minimal changes to existing LLM apps and is compatible with major platforms like ChatGPT for various use cases.\u003c/p\u003e\n\u003ch2 id=\"1-introduction\"\u003e\n \u003cspan\u003e1. Introduction\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#1-introduction\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eInstruction-following language models, such as GPT4 \u003csup id=\"fnref:1\"\u003e\u003ca href=\"#fn:1\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e1\u003c/a\u003e\u003c/sup\u003e, have achieved remarkable success in natural language understanding \u003csup id=\"fnref:2\"\u003e\u003ca href=\"#fn:2\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e2\u003c/a\u003e\u003c/sup\u003e, commonsense reasoning \u003csup id=\"fnref:3\"\u003e\u003ca href=\"#fn:3\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e3\u003c/a\u003e\u003c/sup\u003e, and open-domain QA \u003csup id=\"fnref:4\"\u003e\u003ca href=\"#fn:4\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e4\u003c/a\u003e\u003c/sup\u003e and continue to drive real world successes. Recent LLM applications have been deployed with planning \u003csup id=\"fnref:5\"\u003e\u003ca href=\"#fn:5\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e5\u003c/a\u003e\u003c/sup\u003e, retrieval \u003csup id=\"fnref:6\"\u003e\u003ca href=\"#fn:6\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e6\u003c/a\u003e\u003c/sup\u003e, and tool utilization \u003csup id=\"fnref:7\"\u003e\u003ca href=\"#fn:7\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e7\u003c/a\u003e\u003c/sup\u003e capabilities, allowing them to connect to external services and automate complex tasks. A notable example is ChatGPT, which provides users with a platform to interact with LLMs equipped with plugins and actions that encompass document retrieval, code execution, and communication with third-party services like Expedia and Zapier. Furthermore, there are a number of independently deployed applications that transform these models into agents capable of performing specific tasks such as web searching \u003csup id=\"fnref:8\"\u003e\u003ca href=\"#fn:8\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e8\u003c/a\u003e\u003c/sup\u003e, education \u003csup id=\"fnref:9\"\u003e\u003ca href=\"#fn:9\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e9\u003c/a\u003e\u003c/sup\u003e, and healthcare assistance \u003csup id=\"fnref:10\"\u003e\u003ca href=\"#fn:10\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e10\u003c/a\u003e\u003c/sup\u003e. We refer to these LLM-integrated applications that allow LLMs to interact with external resources as \u003cem\u003eLLM apps\u003c/em\u003e.\u003c/p\u003e\n\u003cp\u003eThere are significant potential security risks when deploying and managing LLMs apps, including indirect prompt injection and insecure output handling, which allow attackers to lead LLMs to generate unexpected outputs, as summarized in OWASP\u0026rsquo;s list of top 10 vulnerabilities found in LLM apps \u003csup id=\"fnref:11\"\u003e\u003ca href=\"#fn:11\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e11\u003c/a\u003e\u003c/sup\u003e. This work focuses on mitigating one specific risk in the list, \u003cstrong\u003eexcessive agency\u003c/strong\u003e, a result of LLM app\u0026rsquo;s ability to interface with external services and perform actions in response to a prompt. Specifically, we tackle the problem where LLM in apps are granted with excessive permission and functionality, and hence over-privileged. In such scenarios, malicious app developers and platforms (e.g. OpenAI) may compromise users\u0026rsquo; privacy and data security by abusing the delegated privileges (e.g. checking user\u0026rsquo;s personal emails), rogue and oblivious models may perform irreversible actions on the shared resources (e.g. creating public Github repos \u003csup id=\"fnref:12\"\u003e\u003ca href=\"#fn:12\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e12\u003c/a\u003e\u003c/sup\u003e), and external attackers may try to access protected resources.\u003c/p\u003e\n\u003cp\u003eWhile LLM apps need privileges for real-world utility, we mitigate the risks of over-privilege in the current LLM app architecture by achieving controlled sharing with Macaroons (i.e., cookies with contextual caveats) and improving transparency and control via a local action proxy with (optionally) human in the loop. These designs enable an architecture that allows users to practically distrust LLM apps by avoiding token sharing, which mitigates excessive permission. Additionally, third-party services, together with LLM apps, can attenuate privilege on resources, thereby reducing excessive functionality.\u003c/p\u003e\n\u003cp\u003eThis paper offers the following contributions:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003eWe develop a security and privacy-sensitive architecture for LLM apps, called LLMacaroon, that reduces the risk of over-privilege in LLM apps. LLMacaroon achieves controlled sharing of resources and improves transparency and control for users.\u003c/li\u003e\n\u003cli\u003eWe demonstrate that LLMacaroon can be practically deployed with acceptable changes to existing infrastructure using prototype LLM apps. We also show its applicability to both self-deployed LLM apps and centralized platforms like OpenAI GPTs.\u003c/li\u003e\n\u003c/ol\u003e\n\u003ch2 id=\"2-background\"\u003e\n \u003cspan\u003e2. Background\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#2-background\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003ch3 id=\"llm-application-vulnerabilities\"\u003e\n \u003cspan\u003eLLM application vulnerabilities.\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#llm-application-vulnerabilities\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eIntegrating API-interfacing ability into the LLM ecosystem exposes additional attack surfaces. Recent works have suggested that plugins introduce attack surfaces between plugins and users, plugins and LLM platform, and between plugins \u003csup id=\"fnref:13\"\u003e\u003ca href=\"#fn:13\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e13\u003c/a\u003e\u003c/sup\u003e. These exposed attack surfaces allow potential malicious plugins to launch attacks targeting users, LLM platforms, and other plugins \u003csup id=\"fnref1:13\"\u003e\u003ca href=\"#fn:13\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e13\u003c/a\u003e\u003c/sup\u003e. Malicious plugins could potentially hijack users\u0026rsquo; machines, hijack the LLM platform, and hijack other plugins \u003csup id=\"fnref2:13\"\u003e\u003ca href=\"#fn:13\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e13\u003c/a\u003e\u003c/sup\u003e.\u003c/p\u003e\n\u003cp\u003eLLM apps can also be compromised with Indirect Prompt Injection. An attacker could inject an adversarial prompt into sources that are likely to be retrieved by the LLM or third-party services \u003csup id=\"fnref:14\"\u003e\u003ca href=\"#fn:14\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e14\u003c/a\u003e\u003c/sup\u003e. For example, if the LLM has web searching capabilities, then the attacker could place prompts on webpages, causing it to be controlled by attackers \u003csup id=\"fnref1:14\"\u003e\u003ca href=\"#fn:14\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e14\u003c/a\u003e\u003c/sup\u003e. This is usually hidden from users because the API calls are executed in the background on remote servers.\u003c/p\u003e\n\u003ch3 id=\"access-control-in-llm-apps\"\u003e\n \u003cspan\u003eAccess Control in LLM Apps.\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#access-control-in-llm-apps\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eTraditional token-based mechanisms are commonly used in LLM apps for permission delegation. Most commonly, this is done either via service-level authentication (e.g., fixed bearer token) to delegate access to the LLM app without linking specific users, or user-level authentication (e.g., OAuth) to delegate user-specific resources. We conducted a pilot study on ChatGPT Plugins. Developers on this platform use manifest files to define the behavior of their plugins. We analyzed the authentication methods used in 866 publicly available plugin manifests (Table 1). Our results show that 17.9% of plugins need user-level authentication for their functionality, and 13.74% conduct service-level authentication, which together constitute a considerable portion of all applications. This result may suggest the prevalence of the need for user-specific authentication in the growing LLM app ecosystem.\u003c/p\u003e\n\u003ctable\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth\u003eMethod\u003c/th\u003e\n \u003cth\u003eCount\u003c/th\u003e\n \u003cth\u003ePercentage\u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd\u003eNone\u003c/td\u003e\n \u003ctd\u003e592\u003c/td\u003e\n \u003ctd\u003e68.36%\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003eOAuth\u003c/td\u003e\n \u003ctd\u003e155\u003c/td\u003e\n \u003ctd\u003e17.9%\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003eBearer\u003c/td\u003e\n \u003ctd\u003e155\u003c/td\u003e\n \u003ctd\u003e13.28%\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003eBasic\u003c/td\u003e\n \u003ctd\u003e4\u003c/td\u003e\n \u003ctd\u003e0.46%\u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eTable 1: Authentication methods from collected 866 ChatGPT Plugin manifests\u003c/p\u003e\n\u003ch2 id=\"3-design-goals-and-threat-model\"\u003e\n \u003cspan\u003e3. Design Goals and Threat Model\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#3-design-goals-and-threat-model\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eOur goal is to restrict the privilege of LLM apps so that malicious or oblivious LLM apps do not compromise user\u0026rsquo;s privacy and data security. Meanwhile, we want to impose minimal change to LLM app\u0026rsquo;s functionalities, where users can exercise \u003cem\u003econtrolled sharing\u003c/em\u003e for delegating specific resources. We first describe the threat model under which we design the system. Then we state the design goals for LLMacaroon. Finally, we present several alternative approaches which we did not implement since they fail to achieve our design goals.\u003c/p\u003e\n\u003ch3 id=\"31-threat-model\"\u003e\n \u003cspan\u003e3.1 Threat Model\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#31-threat-model\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eTo capture a stronger security notion, we assume the LLM app \u0026ndash; both its components like the model itself and the app\u0026rsquo;s providers \u0026ndash; might be untrustworthy. For instance, the LLM app might use a user\u0026rsquo;s shared token to stealthily inspect their personal documents, or perform malicious actions like deleting emails in response to a user\u0026rsquo;s normal prompt. This is a realistic assumption since LLM-integrated are shown to be unreliable under certain scenarios including indirect prompt injection \u003csup id=\"fnref:15\"\u003e\u003ca href=\"#fn:15\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e15\u003c/a\u003e\u003c/sup\u003e, and LLM app platforms like ChatGPT, being overprivileged hubs of user tokens, are particularly susceptible to security breaches.\u003c/p\u003e\n\u003cp\u003eOn the other hand, we trust end-users of the LLM app, who do not expect actions to their own detriment. Therefore, jail breaking \u003csup id=\"fnref:16\"\u003e\u003ca href=\"#fn:16\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e16\u003c/a\u003e\u003c/sup\u003e \u003csup id=\"fnref:17\"\u003e\u003ca href=\"#fn:17\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e17\u003c/a\u003e\u003c/sup\u003e and general prompt injection attacks are out of scope of this work. However, we expect standard attackers of the system who want to access resources outside their permission (e.g. other users\u0026rsquo; documents).\u003c/p\u003e\n\u003ch3 id=\"32-design-goals\"\u003e\n \u003cspan\u003e3.2 Design Goals\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#32-design-goals\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003ch4 id=\"security-and-privacy\"\u003e\n \u003cspan\u003eSecurity and Privacy.\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#security-and-privacy\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h4\u003e\n\u003cp\u003eOur primary goal is to reduce the long-lived privileges in LLM apps and only grant ephemeral access to necessary resources while the system is in use. This is to ensure that malicious LLMs and their providers do not abuse the privilege for extraneous purposes that threaten users\u0026rsquo; privacy and data security. Our additional focus is to give users transparency and control so that they can exercise their discretion while interacting with the LLM apps. This is to provide more fine-grained control during the period when LLM apps have ephemeral privileges. Finally, the design should not open up new vulnerabilities for the standard security model of LLM apps and privileges of users should be well isolated.\u003c/p\u003e\n\u003ch4 id=\"functionality\"\u003e\n \u003cspan\u003eFunctionality.\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#functionality\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h4\u003e\n\u003cp\u003eThe design to reduce long-lived privileges in LLM apps must not sacrifice the existing functionalities of LLM apps and their development flexibility. For instance, LLM apps should still be able to communicate to external services and perform the same set of actions spanning from knowledge retrieval to tool usage. Moreover, LLM apps should be able to implement user-specific business logics and attenuate privileges in a fine-grained manner for different users.\u003c/p\u003e\n\u003ch3 id=\"33-alternative-designs\"\u003e\n \u003cspan\u003e3.3 Alternative Designs\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#33-alternative-designs\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003ch4 id=\"offline-llm-apps\"\u003e\n \u003cspan\u003eOffline LLM Apps.\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#offline-llm-apps\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h4\u003e\n\u003cp\u003eOffline LLM apps provide the strongest privacy and security guarantees to the users if no data leaves the local execution environment (e.g. privateGPT \u003csup id=\"fnref:18\"\u003e\u003ca href=\"#fn:18\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e18\u003c/a\u003e\u003c/sup\u003e. However, completely offline apps sacrifice usability and practical convenience for general users, and it may hinder the service provider\u0026rsquo;s flexibility to customize user experiences, which often requires granting different sets of privileges, violating the functionality design goal.\u003c/p\u003e\n\u003ch4 id=\"application-end-human-in-the-loop\"\u003e\n \u003cspan\u003eApplication-end human in the loop.\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#application-end-human-in-the-loop\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h4\u003e\n\u003cp\u003eApplication platforms like ChatGPT provide an option to prompt for user confirmation before the LLM app performs consequential behaviors \u003csup id=\"fnref:19\"\u003e\u003ca href=\"#fn:19\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e19\u003c/a\u003e\u003c/sup\u003e. This reduces the chance of accidental execution of unwanted actions. However, this mechanism does not offer security guarantee, especially when the application platform is untrusted. Hence, it is not sufficient for our security goal.\u003c/p\u003e\n\u003ch4 id=\"privilege-minimization\"\u003e\n \u003cspan\u003ePrivilege minimization.\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#privilege-minimization\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h4\u003e\n\u003cp\u003ePrivilege minimization in LLM apps involves granting only essential access levels for functionality. Traditional methods often grant permissions without adequate context and fail to restrict access based on time, allowing unnecessary access to sensitive data even when not actively using the app. Therefore, we consider traditional privilege minimization insufficient, and advancements in this line of research should be orthogonal to our design. Our approach uses macaroon to contextually attenuate privileges, which offers better flexibility.\u003c/p\u003e\n\u003ch2 id=\"4-llmacaroon-designs\"\u003e\n \u003cspan\u003e4. LLMacaroon Designs\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#4-llmacaroon-designs\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eTo address the threat model and achieve the design goals outlined in Section 3, we propose LLMacaroon, a novel architecture that minimizes the risks of excessive agency in LLM apps and improves user transparency and control, thereby providing better security against threats from untrusted platforms and app providers when models are delegated to handle sensitive resources. LLMacaroon assumes typical LLM app setup where users interact with remotely hosted app that performs actions on behalf of the user by calling third-party destination services. We summarize the design and discuss the individual components in this section.\u003c/p\u003e\n\u003ch3 id=\"design-overview\"\u003e\n \u003cspan\u003eDesign Overview.\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#design-overview\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eUnlike the standard paradigm, where users delegate sensitive resources to a LLM app and the application performs actions autonomously, we make several paradigm shifts. These changes remove long-lived privileges from the LLM app while allowing them to conduct controlled sharing via contextual attenuation of actions.\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003eRather than storing tokens with actual privileges, LLM apps only hold macaroons with partially fulfilled third-party caveats from destination services (4.1). This requires all actions to go through a local proxy, which gathers necessary discharges.\u003c/li\u003e\n\u003cli\u003eThe LLM app sends the intended action to a trusted local action proxy (4.2) for execution. The proxy gathers third-party authorization and user consent before forwarding actions to the destination service. The LLM app cannot perform actions when the proxy is offline, guaranteeing temporal access restriction.\u003c/li\u003e\n\u003cli\u003eBoth the LLM app and third-party services may additionally attenuate the privilege on macaroons by adding first-party caveats to confine the functionality of the actions. Destination services contextually verify all caveats before executing the incoming actions (4.3).\u003c/li\u003e\n\u003c/ol\u003e\n\n\n\n\n\n\n\n\n\n\n\u003cfigure\u003e\n \u003cimg src=\"/llmacaroon/seq.svg\" alt=\"Figure 1: Sequence diagram illustrating the LLMacaroon-based security protocol for LLM applications.\" loading=\"lazy\" decoding=\"async\"\u003e\n \n\u003c/figure\u003e\n\n\u003ch3 id=\"41-macaroon-construction\"\u003e\n \u003cspan\u003e4.1 Macaroon Construction\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#41-macaroon-construction\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eLLMacaroon employs macaroons for controlled sharing of resources among principals in an LLM app system. The destination service delegates resources on its server for LLM apps to use by minting macaroons and adding first-party caveats that outline the access patterns, which often restrict user identity, resource URI, and time limit. The destination service must also add a third-party caveat to be fulfilled by discharges from authorization services. The service adds caveats using a chained-HMAC construction, whose integrity can be proved with the destination service\u0026rsquo;s secret key. The construction of a macaroon can be initiated by the LLM app at the start of a session, such that it holds the partially fulfilled macaroon during the session and attach it to a request to the user\u0026rsquo;s local proxy for an action execution.\u003c/p\u003e\n\u003ch4 id=\"transmission\"\u003e\n \u003cspan\u003eTransmission.\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#transmission\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h4\u003e\n\u003cp\u003eAfter its construction, the macaron attaches to requests to be shared between services. Specifically, it can be serialized as a string and attached as an HTTP header value or as a cookie if the proxy implements methods for handling cookie semantics. The first hop after a macaroon\u0026rsquo;s construction is the LLM app, which may temporarily store the macaroon on a session basis or a request basis and attenuate it accordingly. The macaroon is forwarded to the proxy and then to the destination service when the LLM app initiates an action.\u003c/p\u003e\n\u003ch4 id=\"application-side-attenuation\"\u003e\n \u003cspan\u003eApplication-Side Attenuation.\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#application-side-attenuation\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h4\u003e\n\u003cp\u003eThe LLM app can also attenuate the macaroon\u0026rsquo;s authority by adding first-party caveats, which is helpful for adding application-specific access control (e.g. a user only see documents with certain tags in a LLM document retrieval app). The attenuation may also prevent misuse of the authority by imposing narrower time and scope limit based on application specific requirements. The LLM app gains knowledge about the supported caveats by inspecting the destination service\u0026rsquo;s specification out-of-band.\u003c/p\u003e\n\u003ch3 id=\"42-local-action-proxy\"\u003e\n \u003cspan\u003e4.2 Local Action Proxy\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#42-local-action-proxy\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eTo perform an action and call a third-party service, the LLM app has to send the request to a trusted local action proxy. A direct call to the destination service will fail because of the unfulfilled third-party caveat. The proxy is responsible for obtaining authority from a third-party authorization service as well as users\u0026rsquo; consent based on a user\u0026rsquo;s policy. The implementation of such proxy can take the form of a browser extension or a local service with web interface, but it carries the following features.\u003c/p\u003e\n\u003ch4 id=\"authorization\"\u003e\n \u003cspan\u003eAuthorization.\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#authorization\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h4\u003e\n\u003cp\u003eIn LLMacaroon, the local action proxy replaces the LLM app\u0026rsquo;s responsibility for obtaining and storing authorizing tokens, thereby minimizing the privilege of LLM apps. On the LLM\u0026rsquo;s first request to the proxy for the delegation of an action, the user is prompted to authenticate with (third-party) authentication service, which returns a discharge to the proxy upon success. The discharge might contain first-party caveats (e.g. to ensure its freshness) and is bound to the main macaron (i.e. sealed) before being sent to the destination service. With the user\u0026rsquo;s consent, the proxy may store the discharge identified by request origins to a local database, analogous to browsers\u0026rsquo; storage of authorization tokens. Subsequent requests to the local proxy may look up the discharge storage to prevent repetitive authentication.\u003c/p\u003e\n\u003ch4 id=\"transparency-and-control\"\u003e\n \u003cspan\u003eTransparency and Control.\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#transparency-and-control\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h4\u003e\n\u003cp\u003eThe proxy provides mechanisms that support transparency and user control in the interaction with LLM apps. This includes supporting users to decide individual action outcomes or specify policies for relaxed automation. Specifically, the proxy 1) presents a transparent log of actions requested by the LLM app as well as the proxy decisions, 2) allows users to customize their authorization preferences, including setting up rules for automatic approvals and specifying conditions under which manual approval is required, and 3) sends real-time notifications to the user in case immediate input is required and allows user to manually intervene and change the decision made by the proxy in real-time. These ensure the user has a transparent view of the proxy behavior and has governance over action requests.\u003c/p\u003e\n\u003ch3 id=\"43-action-verification\"\u003e\n \u003cspan\u003e4.3 Action Verification\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#43-action-verification\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eIf it is able to collect authorizing discharges and either automated or manual approval, the proxy forwards the request to the destination service with the main macaroon, as well as the collected third-party authenticating macaroons. Before processing the request, the destination service verifies all caveats attached to it. The service rejects the requests if the verification is unsuccessful.\u003c/p\u003e\n\u003ch4 id=\"contextual-verification\"\u003e\n \u003cspan\u003eContextual Verification.\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#contextual-verification\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h4\u003e\n\u003cp\u003eThe destination service performs a contextual verification process, which involves assessing the validity of the macaroon in the context of the current request. This means checking not only the integrity and authenticity of the macaroon itself but also ensuring that the caveats are relevant and appropriate for the specific request being made. For instance, if a macaroon has a time limit caveat, the service checks the current time against this limit. Similarly, if there are resource-specific caveats, the service confirms that the request aligns with these constraints. This maintains a secure and context-aware authorization system.\u003c/p\u003e\n\u003ch3 id=\"44-implementation\"\u003e\n \u003cspan\u003e4.4 Implementation\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#44-implementation\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003ch4 id=\"services\"\u003e\n \u003cspan\u003eServices.\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#services\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h4\u003e\n\u003cp\u003eWe provided a Python library for adding macaroon-based access control to services. The library allows services to integrate macaroon authorization with minimal changes by simply adding a middleware component and providing a set of validators for caveats. We leverage the existing pymacaroon library for macaroon construction and verification and integrate it with the FastAPI framework for building the services.\u003c/p\u003e\n\u003cp\u003eThe library also includes a domain-specific language (DSL) for defining caveats and creating validators, supporting basic data types like strings, numbers, and dates as well as binary operators for constraints. The DSL enables services to flexibly specify caveats with placeholders that can be contextually verified at runtime so that developers do not need to repeat parsing and expression evaluation code. These toolings give both the flexibility and the convenience for services to control sharing. The formal definition of DSL is in Appendix A.\u003c/p\u003e\n\u003ch4 id=\"proxy\"\u003e\n \u003cspan\u003eProxy\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#proxy\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h4\u003e\n\u003cp\u003eWe implemented a functional local action proxy server with features outlined in 4.2. It handles obtaining third-party discharges in exchange for user authentication and conditionally stores them in a local database indexed by requesting origins. The proxy server exposes a single endpoint for forwarding actions to destination services. If the request is not blocked by authentication, we stream the body to reduce latency.\u003c/p\u003e\n\u003cp\u003eThe proxy runs a Svelte-based web interface that runs on a separate process that communicates actions and decisions with the server over a WebSocket. This front-end component allows users to view action logs, set authorization preferences, manually approve requests, and override automated decisions. Together, the proxy server and web interface deliver transparency and control.\u003c/p\u003e\n\u003ch4 id=\"llm-apps\"\u003e\n \u003cspan\u003eLLM Apps.\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#llm-apps\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h4\u003e\n\u003cp\u003eFor rapid prototyping of standard LLM apps, we leverage OpenAI\u0026rsquo;s platform, GPTs, which offers easy integration of actions. We wrap existing service endpoints as parameters to the proxy server\u0026rsquo;s forwarding endpoint. The LLM apps access allowed actions and services but delegate execution to the local proxy.\u003c/p\u003e\n\u003cp\u003eMacaroons are supplied to LLM apps as static authorization tokens configured through OpenAI\u0026rsquo;s interface. We attach them via a custom \u0026ldquo;x-macaroon\u0026rdquo; header that services recognize for verification. The macaroons contain pre-defined caveats equivalent to the ones programmatically added by self-hosted LLM app implementations.\u003c/p\u003e\n\u003ch3 id=\"45-examples\"\u003e\n \u003cspan\u003e4.5 Examples\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#45-examples\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eWe implemented several LLM apps that builds upon the LLMacaroon architecture.\u003c/p\u003e\n\u003ch4 id=\"manipulating-sensitive-resources-email-assistant\"\u003e\n \u003cspan\u003eManipulating sensitive resources (Email Assistant).\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#manipulating-sensitive-resources-email-assistant\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h4\u003e\n\u003cp\u003eWe created a GPT plug-in called GPT email service using OpenAI\u0026rsquo;s plug-in building tool. The app reads and sends emails on behalf of the user. This app requires reading users\u0026rsquo; sensitive data (emails) and performing irreversible actions such as sending emails. We also created each of our design components described in 4: a proxy, an authentication service, and a simple email service. We assume the LLM email app has already obtained the undischarged Macaroon from the simple email service. This macaroon is then passed along as a custom API key. We then prompted the GPT email app to get the user\u0026rsquo;s last email. The app requests our proxy service with the undischarged macaroon. The proxy receives the macaroon, obtains the discharge from the third-party service, and sends it to the simple email service. The email service verifies the signature, all the caveats, and all the discharges. After they are verified, the email service calls the read method and sends the result of read - which is the user\u0026rsquo;s latest email - back to the proxy. The proxy then sends the response back to the GPT email app. Finally, the GPT email app is able to obtain the user\u0026rsquo;s latest email and successfully outputs the subject and content information to ChatGPT\u0026rsquo;s UI.\u003c/p\u003e\n\u003ch4 id=\"retrieval-augmented-generation-with-shared-data-documents-search\"\u003e\n \u003cspan\u003eRetrieval Augmented Generation with Shared Data (Documents Search).\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#retrieval-augmented-generation-with-shared-data-documents-search\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h4\u003e\n\u003cp\u003eAnother common class of applications delegates documents to the LLM for retrieval, so that the LLM app conducts generation with grounded, up-to-date knowledge. We implemented an LLM app with RAG ability using the LLMacaroon design. The LLM is given an API to query relevant data from a third-party service that owns a Pinecone vector database and performs embedding similarity search with LangChain. We then protect the API by using Macaroon for authentication and controlled sharing. Specifically, the service may allow access to only a part of the database by adding caveats that check document labels. This also enables cooperative LLM apps to refine the authority given to users (e.g., with a time limit), but malicious apps cannot amplify the authority.\u003c/p\u003e\n\u003ch4 id=\"remote-code-execution-python-interpreter\"\u003e\n \u003cspan\u003eRemote Code execution (Python Interpreter).\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#remote-code-execution-python-interpreter\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h4\u003e\n\u003cp\u003eFinally, we implemented an example LLM app that can execute Python code on users\u0026rsquo; delegated personal computing environment on a prototype service (similar to ECS). We note that fully allowing an LLM app to execute code on a personal computing environment is risky, even when the environment is sandboxed. For example, a malicious LLM app might attack the availability of the computing resource by overloading it. With LLMacaroon, we improved the prototype LLM app by allowing it to request the execution of Python snippets on a third-party service, where the user owns a Docker container. The request goes through the action proxy, and the user may inspect the code before execution.\u003c/p\u003e\n\u003ch2 id=\"5-evaluation\"\u003e\n \u003cspan\u003e5. Evaluation\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#5-evaluation\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003ch3 id=\"51-functional-evaluation\"\u003e\n \u003cspan\u003e5.1 Functional Evaluation\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#51-functional-evaluation\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eExtensive tests are done on the prototype apps to ensure their functionality and correctness. In this section, we detail our functional evaluation on the email assistant app. We first prompted the GPT email app to read the user\u0026rsquo;s latest email by sending a request with the Macaroon given by our email service to our architecture. The request shows that GPT email app successfully obtained the user\u0026rsquo;s latest email.\u003c/p\u003e\n\u003cp\u003eWe also prompted the GPT email app to send an email using our architecture. Sending an email goes through the same process as reading the latest email, except that it calls the email service\u0026rsquo; send function instead of the read function. Although the context is different, we didn\u0026rsquo;t need to write a new validator because our DSL described in 4.4 is able to contextually verify the caveats. The end result is that the GPT email app is able to email one of the researcher\u0026rsquo;s mailbox. The full interaction is in Appendix B.\u003c/p\u003e\n\u003cp\u003eWe then shut down the third-party authentication service and prompted the GPT email app to read the user\u0026rsquo;s last email. This is to simulate a direct API call from the GPT email app to the email service. The email service is unable to verify the discharges and returns an error. The result is that the GPT email app is unable to get the user\u0026rsquo;s latest email.\u003c/p\u003e\n\u003cp\u003eGPT email app\u0026rsquo;s inability to directly perform email actions demonstrates the principle of least privilege. The privilege to perform critical email actions is now returned to the user.\u003c/p\u003e\n\u003ch3 id=\"52-performance-evaluation\"\u003e\n \u003cspan\u003e5.2 Performance Evaluation\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#52-performance-evaluation\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eThe performance overhead of Macaroon operations has already been discussed by Birgisson et al. \u003csup id=\"fnref:20\"\u003e\u003ca href=\"#fn:20\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e20\u003c/a\u003e\u003c/sup\u003e. To test the performance overhead for our architecture, we conduct an end-to-end application performance test on a no-op prototype apps.\u003c/p\u003e\n\u003cp\u003eWe measure the total time it took for each operation with respect to the number of caveats attenuated. We measured the proxy overhead, discharge overhead, macaroon verification overhead, and overall architecture overhead. The proxy overhead includes the discharge overhead. Proxy overhead is measured by starting a timer when the request is received and stopping the timer when the request is about to be routed to the email service server. We obtained the overall architecture overhead by taking the sum of proxy overhead and macaroon validation overhead. This is because the overhead of routing and sending HTTP requests to the email service server is also done even without using our architecture. Therefore, the routing of HTTP requests is not included. Figure 2 shows our performance evaluation of the GPT email app.\u003c/p\u003e\n\n\n\n\n\n\n\n\n\n\n\u003cfigure\u003e\n \u003cimg src=\"/llmacaroon/overhead_graph.png\" alt=\"Figure 2: Performance overhead in seconds with respect to the number of caveats added\" loading=\"lazy\" decoding=\"async\" width=\"1718\" height=\"1101\"\u003e\n \n\u003c/figure\u003e\n\n\u003cp\u003eIn Figure 2, we present a graph of comprehensive analysis of performance. The y-axis represents the average time measured in seconds, and the x-axis denotes the number of caveats in a macaroon. The data depicted in this graph indicates an almost constant architecture overhead of 0.13 seconds. The main contributor to this overhead is the routing of macaroon to the third-party discharge service. The discharge overhead contributed to 95.15% of the overall architecture overhead. We foresee that most GPT plugins would typically utilize only a few caveats. Consequently, the security enhancements provided by LLMacaroon are achieved with a minimal impact on performance, as demonstrated by the overhead figures for a modest number of caveats.\u003c/p\u003e\n\u003ch2 id=\"6-discussion-and-limitation\"\u003e\n \u003cspan\u003e6. Discussion and Limitation\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#6-discussion-and-limitation\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eThis paper introduces a novel architecture, LLMacaroon, designed to enhance the security and privacy of Large Language Model (LLM) applications. LLMacaroon employs macaroons for controlled sharing of resources and incorporates a local action proxy to increase user transparency and control. However, we note some limitations that can be studied in future works.\u003c/p\u003e\n\u003ch3 id=\"usability\"\u003e\n \u003cspan\u003eUsability\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#usability\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eThe introduction of the local action proxy is a step towards handing back transparency and control to users, which opens up opportunities for more effective user-centric designs. This proxy can be implemented in various forms, such as a browser extension or a local web application, allowing users to exercise more direct control over their interactions with LLM applications.\u003c/p\u003e\n\u003cp\u003eHowever, the effectiveness of these implementations in real-world scenarios remains unexplored, as our current work does not focus on interaction design. Future work should investigate the impact of different proxy implementations on user experience and the effectiveness of user control in practical settings, possibly through intelligent, context-aware interfaces that reduce the cognitive load on users. Additionally, exploring decentralized approaches for the action proxy could further enhance trust and security in the system, as we shift trust from LLM apps to the proxy.\u003c/p\u003e\n\u003ch3 id=\"adoption\"\u003e\n \u003cspan\u003eAdoption\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#adoption\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eFor LLM applications, adoption of LLMacaroon does not necessitate major changes, preserving their existing functionality. However, services interfacing with these applications need modifications to adhere to the additional security constraints introduced by LLMacaroon. From the user\u0026rsquo;s perspective, installing an action proxy can be optional, allowing users to opt for the existing paradigm if they choose. This flexibility supports a gradual transition to the new architecture, reducing barriers to adoption. Future studies should explore the user acceptance and adoption rates of the action proxy, examining factors that influence users\u0026rsquo; decisions to adopt or reject this new layer of control and security.\u003c/p\u003e\n\u003ch3 id=\"broader-implications\"\u003e\n \u003cspan\u003eBroader Implications\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#broader-implications\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eLLMacaroon may extend beyond LLM applications, potentially influencing the design of other types of applications that handle sensitive user data or require controlled resource sharing. The concepts introduced in this architecture could inspire new approaches to user-centric privacy and security across various domains.\u003c/p\u003e\n\u003ch2 id=\"acknowledgements\"\u003e\n \u003cspan\u003eAcknowledgements\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#acknowledgements\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eWe want to thank Professor Earlence Fernandes and TA Alisha Ukani for many helpful discussions.\u003c/p\u003e\n\u003ch2 id=\"appendix\"\u003e\n \u003cspan\u003eAppendix\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#appendix\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003ch3 id=\"a-formal-definition\"\u003e\n \u003cspan\u003eA. Formal Definition\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#a-formal-definition\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\n\n\n\n\n\n\n\n\n\n\u003cfigure\u003e\n \u003cimg src=\"/llmacaroon/formal-defination.png\" alt=\"Figure 3: Formal definition of DSL mentioned in 4.4\" loading=\"lazy\" decoding=\"async\" width=\"751\" height=\"554\"\u003e\n \n\u003c/figure\u003e\n\n\u003ch3 id=\"b-gpt-email-app-interaction\"\u003e\n \u003cspan\u003eB. GPT Email APP Interaction\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#b-gpt-email-app-interaction\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\n\n\n\n\n\n\n\n\n\n\u003cfigure\u003e\n \u003cimg src=\"/llmacaroon/interaction1.png\" alt=\"Figure 4: Asking GPT email app to get the user\u0026rsquo;s latest email\" loading=\"lazy\" decoding=\"async\" width=\"879\" height=\"451\"\u003e\n \n\u003c/figure\u003e\n\n\n\n\n\n\n\n\n\n\n\n\u003cfigure\u003e\n \u003cimg src=\"/llmacaroon/interaction2.png\" alt=\"Figure 5: Asking GPT email app to send an email\" loading=\"lazy\" decoding=\"async\" width=\"912\" height=\"352\"\u003e\n \n\u003c/figure\u003e\n\n\n\n\n\n\n\n\n\n\n\n\u003cfigure\u003e\n \u003cimg src=\"/llmacaroon/interaction3.png\" alt=\"Figure 6: Asking GPT email app to get the user\u0026rsquo;s latest email without discharging the Macaroon\" loading=\"lazy\" decoding=\"async\" width=\"843\" height=\"723\"\u003e\n \n\u003c/figure\u003e\n\n\u003ch3 id=\"c-proxy-interface\"\u003e\n \u003cspan\u003eC. Proxy Interface\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#c-proxy-interface\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\n\n\n\n\n\n\n\n\n\n\u003cfigure\u003e\n \u003cimg src=\"/llmacaroon/screenshot.png\" alt=\"Figure 7: Proxy interface. Users inspect and manage incoming requests and review logs.\" loading=\"lazy\" decoding=\"async\" width=\"1488\" height=\"1586\"\u003e\n \n\u003c/figure\u003e\n\n\u003cp\u003eHere are the references converted into footnotes using Markdown syntax:\u003c/p\u003e\n\u003cdiv class=\"footnotes\" role=\"doc-endnotes\"\u003e\n\u003chr\u003e\n\u003col\u003e\n\u003cli id=\"fn:1\"\u003e\n\u003cp\u003eOpenAI. \u0026ldquo;GPT-4 Technical Report,\u0026rdquo; March 27, 2023. \n\n\n\n\n\n\u003ca href=\"http://arxiv.org/abs/2303.08774\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttp://arxiv.org/abs/2303.08774\u003c/a\u003e\n.\u0026#160;\u003ca href=\"#fnref:1\" class=\"footnote-backref\" role=\"doc-backlink\"\u003e\u0026#x21a9;\u0026#xfe0e;\u003c/a\u003e\u003c/p\u003e\n\u003c/li\u003e\n\u003cli id=\"fn:2\"\u003e\n\u003cp\u003eWolf, Thomas, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, et al. \u0026ldquo;Transformers: State-of-the-Art Natural Language Processing.\u0026rdquo; In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, 38–45. Online: Association for Computational Linguistics, 2020. \n\n\n\n\n\n\u003ca href=\"https://aclanthology.org/2020.emnlp-demos.6\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttps://aclanthology.org/2020.emnlp-demos.6\u003c/a\u003e\n.\u0026#160;\u003ca href=\"#fnref:2\" class=\"footnote-backref\" role=\"doc-backlink\"\u003e\u0026#x21a9;\u0026#xfe0e;\u003c/a\u003e\u003c/p\u003e\n\u003c/li\u003e\n\u003cli id=\"fn:3\"\u003e\n\u003cp\u003eTalmor, Alon, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. \u0026ldquo;CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge,\u0026rdquo; March 15, 2019. \n\n\n\n\n\n\u003ca href=\"http://arxiv.org/abs/1811.00937\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttp://arxiv.org/abs/1811.00937\u003c/a\u003e\n.\u0026#160;\u003ca href=\"#fnref:3\" class=\"footnote-backref\" role=\"doc-backlink\"\u003e\u0026#x21a9;\u0026#xfe0e;\u003c/a\u003e\u003c/p\u003e\n\u003c/li\u003e\n\u003cli id=\"fn:4\"\u003e\n\u003cp\u003eDunn, Matthew, Levent Sagun, Mike Higgins, V. Ugur Guney, Volkan Cirik, and Kyunghyun Cho. \u0026ldquo;SearchQA: A New Q\u0026amp;A Dataset Augmented with Context from a Search Engine,\u0026rdquo; June 11, 2017. \n\n\n\n\n\n\u003ca href=\"http://arxiv.org/abs/1704.05179\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttp://arxiv.org/abs/1704.05179\u003c/a\u003e\n.\u0026#160;\u003ca href=\"#fnref:4\" class=\"footnote-backref\" role=\"doc-backlink\"\u003e\u0026#x21a9;\u0026#xfe0e;\u003c/a\u003e\u003c/p\u003e\n\u003c/li\u003e\n\u003cli id=\"fn:5\"\u003e\n\u003cp\u003eYao, Shunyu, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. \u0026ldquo;Tree of Thoughts: Deliberate Problem Solving with Large Language Models,\u0026rdquo; December 3, 2023. \n\n\n\n\n\n\u003ca href=\"http://arxiv.org/abs/2305.10601\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttp://arxiv.org/abs/2305.10601\u003c/a\u003e\n.\u0026#160;\u003ca href=\"#fnref:5\" class=\"footnote-backref\" role=\"doc-backlink\"\u003e\u0026#x21a9;\u0026#xfe0e;\u003c/a\u003e\u003c/p\u003e\n\u003c/li\u003e\n\u003cli id=\"fn:6\"\u003e\n\u003cp\u003eLewis, Patrick, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, et al. \u0026ldquo;Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks.\u0026rdquo; In Advances in Neural Information Processing Systems, 33:9459–74. Curran Associates, Inc., 2020. \n\n\n\n\n\n\u003ca href=\"https://proceedings.neurips.cc/paper/2020/hash/6b493230205f780e1bc26945df7481e5-Abstract.html\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttps://proceedings.neurips.cc/paper/2020/hash/6b493230205f780e1bc26945df7481e5-Abstract.html\u003c/a\u003e\n.\u0026#160;\u003ca href=\"#fnref:6\" class=\"footnote-backref\" role=\"doc-backlink\"\u003e\u0026#x21a9;\u0026#xfe0e;\u003c/a\u003e\u003c/p\u003e\n\u003c/li\u003e\n\u003cli id=\"fn:7\"\u003e\n\u003cp\u003eSchick, Timo, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. \u0026ldquo;Toolformer: Language Models Can Teach Themselves to Use Tools,\u0026rdquo; February 9, 2023. \n\n\n\n\n\n\u003ca href=\"http://arxiv.org/abs/2302.04761\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttp://arxiv.org/abs/2302.04761\u003c/a\u003e\n.\u0026#160;\u003ca href=\"#fnref:7\" class=\"footnote-backref\" role=\"doc-backlink\"\u003e\u0026#x21a9;\u0026#xfe0e;\u003c/a\u003e\u003c/p\u003e\n\u003c/li\u003e\n\u003cli id=\"fn:8\"\u003e\n\u003cp\u003e\u0026ldquo;Bing Chat | Microsoft Edge.\u0026rdquo; Accessed December 11, 2023. \n\n\n\n\n\n\u003ca href=\"https://www.microsoft.com/en-us/edge/features/bing-chat\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttps://www.microsoft.com/en-us/edge/features/bing-chat\u003c/a\u003e\n.\u0026#160;\u003ca href=\"#fnref:8\" class=\"footnote-backref\" role=\"doc-backlink\"\u003e\u0026#x21a9;\u0026#xfe0e;\u003c/a\u003e\u003c/p\u003e\n\u003c/li\u003e\n\u003cli id=\"fn:9\"\u003e\n\u003cp\u003e\u0026ldquo;Introducing Duolingo Max, a learning experience powered by GPT-4.\u0026rdquo; Duolingo Blog, March 14, 2023. \n\n\n\n\n\n\u003ca href=\"https://blog.duolingo.com/duolingo-max/\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttps://blog.duolingo.com/duolingo-max/\u003c/a\u003e\n.\u0026#160;\u003ca href=\"#fnref:9\" class=\"footnote-backref\" role=\"doc-backlink\"\u003e\u0026#x21a9;\u0026#xfe0e;\u003c/a\u003e\u003c/p\u003e\n\u003c/li\u003e\n\u003cli id=\"fn:10\"\u003e\n\u003cp\u003eClusmann, Jan, Fiona R. Kolbinger, Hannah Sophie Muti, Zunamys I. Carrero, Jan-Niklas Eckardt, Narmin Ghaffari Laleh, Chiara Maria Lavinia Löffler, et al. \u0026ldquo;The Future Landscape of Large Language Models in Medicine.\u0026rdquo; Communications Medicine 3, no. 1 (October 10, 2023): 1–8. \n\n\n\n\n\n\u003ca href=\"https://doi.org/10.1038/s43856-023-00370-1\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttps://doi.org/10.1038/s43856-023-00370-1\u003c/a\u003e\n.\u0026#160;\u003ca href=\"#fnref:10\" class=\"footnote-backref\" role=\"doc-backlink\"\u003e\u0026#x21a9;\u0026#xfe0e;\u003c/a\u003e\u003c/p\u003e\n\u003c/li\u003e\n\u003cli id=\"fn:11\"\u003e\n\u003cp\u003eWilson, Steve, and others. \u0026ldquo;OWASP Top 10 for Large Language Models (LLMs) - 2023,\u0026rdquo; 2023. \n\n\n\n\n\n\u003ca href=\"https://owasp.org/www-project-top-10-for-large-language-model-applications\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttps://owasp.org/www-project-top-10-for-large-language-model-applications\u003c/a\u003e\n.\u0026#160;\u003ca href=\"#fnref:11\" class=\"footnote-backref\" role=\"doc-backlink\"\u003e\u0026#x21a9;\u0026#xfe0e;\u003c/a\u003e\u003c/p\u003e\n\u003c/li\u003e\n\u003cli id=\"fn:12\"\u003e\n\u003cp\u003e\n\n\n\n\n\n\u003ca href=\"https://www.reddit.com/r/OpenAI/comments/146xl6u/this_is_scary_posting_stuff_by_itself/\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003eLink to Reddit post\u003c/a\u003e\n\u0026#160;\u003ca href=\"#fnref:12\" class=\"footnote-backref\" role=\"doc-backlink\"\u003e\u0026#x21a9;\u0026#xfe0e;\u003c/a\u003e\u003c/p\u003e\n\u003c/li\u003e\n\u003cli id=\"fn:13\"\u003e\n\u003cp\u003eIqbal, Umar, Tadayoshi Kohno, and Franziska Roesner. \u0026ldquo;LLM Platform Security: Applying a Systematic Evaluation Framework to OpenAI\u0026rsquo;s ChatGPT Plugins.\u0026rdquo; 2023. \n\n\n\n\n\n\u003ca href=\"https://llm-platform-security.github.io/chatgpt-plugin-eval\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttps://llm-platform-security.github.io/chatgpt-plugin-eval\u003c/a\u003e\n.\u0026#160;\u003ca href=\"#fnref:13\" class=\"footnote-backref\" role=\"doc-backlink\"\u003e\u0026#x21a9;\u0026#xfe0e;\u003c/a\u003e\u0026#160;\u003ca href=\"#fnref1:13\" class=\"footnote-backref\" role=\"doc-backlink\"\u003e\u0026#x21a9;\u0026#xfe0e;\u003c/a\u003e\u0026#160;\u003ca href=\"#fnref2:13\" class=\"footnote-backref\" role=\"doc-backlink\"\u003e\u0026#x21a9;\u0026#xfe0e;\u003c/a\u003e\u003c/p\u003e\n\u003c/li\u003e\n\u003cli id=\"fn:14\"\u003e\n\u003cp\u003eGreshake, Kai, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, and Thorsten Holz. \u0026ldquo;Not what you\u0026rsquo;ve signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection.\u0026rdquo; Unknown Journal, 2023. Saarland University, Sequire Technology GmbH, CISPA Helmholtz Center for Information Security.\u0026#160;\u003ca href=\"#fnref:14\" class=\"footnote-backref\" role=\"doc-backlink\"\u003e\u0026#x21a9;\u0026#xfe0e;\u003c/a\u003e\u0026#160;\u003ca href=\"#fnref1:14\" class=\"footnote-backref\" role=\"doc-backlink\"\u003e\u0026#x21a9;\u0026#xfe0e;\u003c/a\u003e\u003c/p\u003e\n\u003c/li\u003e\n\u003cli id=\"fn:15\"\u003e\n\u003cp\u003eGreshake, Kai, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. \u0026ldquo;Not What You\u0026rsquo;ve Signed up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection,\u0026rdquo; 2023.\u0026#160;\u003ca href=\"#fnref:15\" class=\"footnote-backref\" role=\"doc-backlink\"\u003e\u0026#x21a9;\u0026#xfe0e;\u003c/a\u003e\u003c/p\u003e\n\u003c/li\u003e\n\u003cli id=\"fn:16\"\u003e\n\u003cp\u003eWei, Alexander, Nika Haghtalab, and Jacob Steinhardt. \u0026ldquo;Jailbroken: How Does LLM Safety Training Fail?,\u0026rdquo; 2023.\u0026#160;\u003ca href=\"#fnref:16\" class=\"footnote-backref\" role=\"doc-backlink\"\u003e\u0026#x21a9;\u0026#xfe0e;\u003c/a\u003e\u003c/p\u003e\n\u003c/li\u003e\n\u003cli id=\"fn:17\"\u003e\n\u003cp\u003eZou, Andy, Zifan Wang, J. Zico Kolter, and Matt Fredrikson. \u0026ldquo;Universal and Transferable Adversarial Attacks on Aligned Language Models,\u0026rdquo; 2023.\u0026#160;\u003ca href=\"#fnref:17\" class=\"footnote-backref\" role=\"doc-backlink\"\u003e\u0026#x21a9;\u0026#xfe0e;\u003c/a\u003e\u003c/p\u003e\n\u003c/li\u003e\n\u003cli id=\"fn:18\"\u003e\n\u003cp\u003eMartínez Toro, Iván, Daniel Gallego Vico, and Pablo Orgaz. \u0026ldquo;PrivateGPT.\u0026rdquo; May 2023. \n\n\n\n\n\n\u003ca href=\"https://github.com/imartinez/privateGPT\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttps://github.com/imartinez/privateGPT\u003c/a\u003e\n.\u0026#160;\u003ca href=\"#fnref:18\" class=\"footnote-backref\" role=\"doc-backlink\"\u003e\u0026#x21a9;\u0026#xfe0e;\u003c/a\u003e\u003c/p\u003e\n\u003c/li\u003e\n\u003cli id=\"fn:19\"\u003e\n\u003cp\u003eOpenAI. \u0026ldquo;Actions in GPTs.\u0026rdquo; \n\n\n\n\n\n\u003ca href=\"https://platform.openai.com/docs/actions\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttps://platform.openai.com/docs/actions\u003c/a\u003e\n.\u0026#160;\u003ca href=\"#fnref:19\" class=\"footnote-backref\" role=\"doc-backlink\"\u003e\u0026#x21a9;\u0026#xfe0e;\u003c/a\u003e\u003c/p\u003e\n\u003c/li\u003e\n\u003cli id=\"fn:20\"\u003e\n\u003cp\u003eBirgisson, Arnar, Joe Gibbs Politz, Úlfar Erlingsson, Ankur Taly, Michael Vrable, and Mark Lentczner. \u0026ldquo;Macaroons: Cookies with Contextual Caveats for Decentralized Authorization in the Cloud,\u0026rdquo; 2013.\u0026#160;\u003ca href=\"#fnref:20\" class=\"footnote-backref\" role=\"doc-backlink\"\u003e\u0026#x21a9;\u0026#xfe0e;\u003c/a\u003e\u003c/p\u003e\n\u003c/li\u003e\n\u003c/ol\u003e\n\u003c/div\u003e","date_modified":"2026-05-30T00:37:17-07:00","date_published":"2023-12-14T22:20:31+08:00","id":"https://www.xiyan.dev/writings/2023/12/14/llmacaroon-re-architecting-platform-for-secure-llm-apps/","summary":"Boosting Large Language Models (LLMs) applications with tools to interact with third-party services enables LLMs to retrieve updated knowledge and perform actions on behalf of users. However, the added capability brings security and privacy risks. In the current paradigm, users delegate potentially sensitive resources to LLM Apps, which makes the platforms overprivileged. For instance, malicious platforms or rogue models can exploit shared email-sending or TAP platform tokens stealthily. We propose LLMacaroon, a practical and secure architecture that distrusts applications for sharing sensitive resources and shifts control back to users. LLMacaroon achieves flexible, controlled sharing via macaroons and improves transparency and control via a local action proxy with optionally human in the loop. We demonstrate that LLMacaroon requires minimal changes to existing LLM apps and is compatible with major platforms like ChatGPT for various use cases.\n","tags":["Security","Privacy","Large Language Models"],"title":"LLMacaroon: Re-architecting Platform for Secure LLM Apps","url":"https://www.xiyan.dev/writings/2023/12/14/llmacaroon-re-architecting-platform-for-secure-llm-apps/"},{"content_html":"\u003ch1 id=\"introduction\"\u003e\n \u003cspan\u003eIntroduction\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#introduction\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h1\u003e\n\u003cp\u003eThe inherently dynamic nature of human communication entails adaptability in both spoken and written discourse, as individuals navigate diverse contexts and audiences. This linguistic malleability, commonly referred to as style, encompasses many of textual attributes including but not limited to formality, politeness, diction, and emotional tenor. Text style transfer (TST), a long-standing endeavor within the field of natural language processing (NLP), seeks to transform specific stylistic attributes while preserving the fundamental meaning of the text. This complex task holds significant potential for enhancing various downstream applications, such as machine translation, where it can facilitate the interpretation of stylistically complex texts; text simplification, which aims to render complex content accessible to a broader audience; and formality transfer, a process that enables the adaptation of text to suit diverse social contexts. The table below illustrates several representative examples of such transformations.\u003c/p\u003e\n\u003ctable\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth style=\"text-align: left\"\u003eOriginal text\u003c/th\u003e\n \u003cth style=\"text-align: left\"\u003eStylized text\u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd style=\"text-align: left\"\u003e(Rude) What the hell is wrong with your attitude?\u003c/td\u003e\n \u003ctd style=\"text-align: left\"\u003e(Polite) Perhaps the question is more about your attitude.\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"text-align: left\"\u003e(Positive) Apple\u0026rsquo;s new chip worths it\u0026rsquo;s price.\u003c/td\u003e\n \u003ctd style=\"text-align: left\"\u003e(Negative) Apple\u0026rsquo;s new chip is to expensive to afford.\u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003ch1 id=\"methodology\"\u003e\n \u003cspan\u003eMethodology\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#methodology\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h1\u003e\n\u003cp\u003eIn this project, I aimed to leverage large pre-trained language models to approach the TST task with transformer models. After experimenting with a naive seq2seq model based on T5\u003csup id=\"fnref:1\"\u003e\u003ca href=\"#fn:1\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e1\u003c/a\u003e\u003c/sup\u003e with a language model head and a Generator-Discriminator network that enables adversarial training\u003csup id=\"fnref:2\"\u003e\u003ca href=\"#fn:2\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e2\u003c/a\u003e\u003c/sup\u003e, I ultimately developed and trained a T5-based conditional generation model with a language model head and a style extractor encoder, which performs significantly better than the GAN model and appears more flexible than the naive seq2seq fine-tuning approach. Proposed by Riley et al., this more recent approach features few-shot tunable targeted restyling and style extraction\u003csup id=\"fnref:3\"\u003e\u003ca href=\"#fn:3\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e3\u003c/a\u003e\u003c/sup\u003e.\u003c/p\u003e\n\u003ch2 id=\"model-architecture\"\u003e\n \u003cspan\u003eModel Architecture\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#model-architecture\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eAs the source code is not openly released, I implemented a \u003ccode\u003eT5ForConditionalGenerationWithExtractor\u003c/code\u003e model by extending the vanilla T5 model from the \u003ccode\u003ehuggingface/transformers\u003c/code\u003e repository. Following the \u003cem\u003eTextSETTR\u003c/em\u003e model architecture, multiple changes were made (e.g., noise generation function, extractor specifications, and hyperparameters) to reproduce the paper\u0026rsquo;s result with limited computational resources. Ultimately, I achieved a working and likely the first third-party implementation of a T5-based text style extraction model. The model definition is as follows.\u003c/p\u003e\n\n\n\n\n\n\n\n\n\n\n\u003cfigure\u003e\n \u003cimg src=\"/model_df.png\" alt=\"Architecture diagram of the T5-based style transfer model, showing the shared encoder-decoder backbone, the added style extractor, and the language model head.\" loading=\"lazy\" decoding=\"async\" width=\"2554\" height=\"4287\"\u003e\n \n\u003c/figure\u003e\n\n\u003cp\u003eIn my implementation, the encoder and decoder are identical to those in the T5-base model, and two additional modules are appended to the encoder-decoder network—a style extractor and a language model head. The style extractor is, in my implementation, a deep copy of the encoder module (weights and biases not tied). Before training, the extractor module is initialized with weights identical to those of the encoder. The language model head is a single-layer feed-forward network projecting the hidden states to the vocabulary size. The last hidden states of the encoder and the style extractor are combined via addition and fed into the decoder\u0026rsquo;s attentions. Under the assumption that adjacent sentences in the same context share the same style, the style extractor will be able to extract the style of the context sentence, as facilitated by a noisy reconstruction task. More details about the training/inference process will be discussed in later sections.\u003c/p\u003e\n\u003ch2 id=\"data-preparation\"\u003e\n \u003cspan\u003eData Preparation\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#data-preparation\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eThe T5 with style extractor model is trained on a subset of the \n\n\n\n\n\n\u003ca href=\"https://nijianmo.github.io/amazon/index.html\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003eAmazon Review Data (2018)\u003c/a\u003e\n dataset. As my experiment setup could not handle entire dataset, I sampled one million lines from the 5-core sub-dataset. The dataset is preprocessed by only taking reviews with ≥ 2 sentences, where each sentence has 30 or more characters. I implemented a PyTorch Dataset class to load the dataset and preprocess the data. For every pair of sentences in the dataset, the first sentence is used as the context and the second sentence as the input. Eventually, the data preprocessor tokenizes the sentences using the original T5 tokenizer, splits the data into training, validation, and testing datasets, and packages all the data loaders into a PyTorch Lightning DataModule for batching and training.\u003c/p\u003e\n\u003ch2 id=\"training\"\u003e\n \u003cspan\u003eTraining\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#training\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eThe training of T5ForConditionalGenerationWithExtractor is unsupervised and facilitated by a reconstruction task. I implemented helper functions to add noise to the input sentence (one for adding noisy tokens (40%) and another for randomly dropping tokens (20%)). Like the original paper\u0026rsquo;s method, I also employed the Noisy Back Translation technique (same amount as the mechanical noises) to improve transfer accuracy. The diagram above illustrates the data flow of the training process. The preceding sentence tokens are used for the context, and the following sentence tokens with Drop/Add + NBT noise are used as the input IDs. With the training goal being to reconstruct the noisy text based on the context style, the ground truth sentences are directly fed into the decoder for a cross-entropy loss atop the linear language model head. For hyperparameters, I used the AdamW optimizer with a learning rate of 1e-3 and PyTorch Lightning\u0026rsquo;s automatic scheduler. The batch size is set to 64, and the maximum number of tokens is set to 32. Since I have access to only one P100 GPU on Google Colab and there are over 332 million trainable parameters, I trained the model for 2 epochs. Hopefully, this is sufficient for a proof of concept.\u003c/p\u003e\n\n\n\n\n\n\n\n\n\n\n\u003cfigure\u003e\n \u003cimg src=\"/eval.png\" alt=\"Few-shot Inference\" loading=\"lazy\" decoding=\"async\" width=\"1987\" height=\"1312\"\u003e\n \n\u003c/figure\u003e\n\n\u003cp\u003eIn inference mode, the strategy illustrated in the figure above is used to transfer text styles on a few-shot basis. First, we choose a target style and a source style similar to the input style. We feed the extractor with exemplars for both styles and obtain the corresponding style embeddings. By subtracting the target style embedding from the source style embedding, we get a style vector pointing towards the target style. In this way, by adding the style direction to the extracted input style, we can obtain the transferred style. The style vector is then summed with the input encoder\u0026rsquo;s final hidden states to get the final embedding, which is to be decoded. After a greedy search for the language model output, we get the predicted output sentence.\u003c/p\u003e\n\u003ch1 id=\"results\"\u003e\n \u003cspan\u003eResults\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#results\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h1\u003e\n\u003ctable\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth style=\"text-align: left\"\u003eSource\u003c/th\u003e\n \u003cth style=\"text-align: left\"\u003eResult\u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd style=\"text-align: left\"\u003eI hereby commit to never purchase anything from this institution in the future.\u003c/td\u003e\n \u003ctd style=\"text-align: left\"\u003eI\u0026rsquo;m hereby gonna never buy from this seller again.\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"text-align: left\"\u003eI couldn’t figure out what the author was trying to say.\u003c/td\u003e\n \u003ctd style=\"text-align: left\"\u003e(Formal) I couldn\u0026rsquo;t figure out what exactly the author was trying to say.\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"text-align: left\"\u003e\u003c/td\u003e\n \u003ctd style=\"text-align: left\"\u003e(Informal) I couldnt figure out what the author was trying to say.\u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003ctable\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth style=\"text-align: left\"\u003ePrompt\u003c/th\u003e\n \u003cth style=\"text-align: left\"\u003eEmotional\u003c/th\u003e\n \u003cth style=\"text-align: left\"\u003eReserved\u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd style=\"text-align: left\"\u003eMy favorite movie is $\\cdots$\u003c/td\u003e\n \u003ctd style=\"text-align: left\"\u003eMy favorate movie is awesome. My Favorate Movie is amazing!\u003c/td\u003e\n \u003ctd style=\"text-align: left\"\u003eMy favorate movie is the movie which is great.\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"text-align: left\"\u003ePlease get $\\cdots$\u003c/td\u003e\n \u003ctd style=\"text-align: left\"\u003ePlease get it!\u003c/td\u003e\n \u003ctd style=\"text-align: left\"\u003ePlease get it. Please do get this.\u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003ctable\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth style=\"text-align: left\"\u003ePrompt\u003c/th\u003e\n \u003cth style=\"text-align: left\"\u003eNegative\u003c/th\u003e\n \u003cth style=\"text-align: left\"\u003ePositive\u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd style=\"text-align: left\"\u003eThe product is $\\cdots$\u003c/td\u003e\n \u003ctd style=\"text-align: left\"\u003eThe product is defective.\u003c/td\u003e\n \u003ctd style=\"text-align: left\"\u003eThe product is good The Product is excellent\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"text-align: left\"\u003eApple watch is $\\cdots$\u003c/td\u003e\n \u003ctd style=\"text-align: left\"\u003eApple watch is useless.\u003c/td\u003e\n \u003ctd style=\"text-align: left\"\u003eApple watch is amazing.\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"text-align: left\"\u003eThe University of California at San Diego is $\\cdots$\u003c/td\u003e\n \u003ctd style=\"text-align: left\"\u003eThe University of California at San Diego is a dead zone.\u003c/td\u003e\n \u003ctd style=\"text-align: left\"\u003eThe University of California at San Diego is beautiful\u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eThe table above showcases some results of the model. For a qualitative evaluation, I tested the model on several tasks. The first group demonstrates the model\u0026rsquo;s performance in the formal ↔ informal transfer task, and all the outputs are generated by conditioning on only 3 exemplars for each style. We can observe that the first result is quite remarkable, and the second result illustrates the flexibility of transfer directions (target style not necessarily ≠ input style)—for the formal → informal direction, notice that it spells \u0026ldquo;couldn\u0026rsquo;t\u0026rdquo; as \u0026ldquo;couldnt,\u0026rdquo; a hallmark of informal language.\u003c/p\u003e\n\u003cp\u003eThe second group of examples shows that besides seq2seq, this model is capable of conditional generation. Provided with a prompt and conditioned on only 3 examples in each style category, the generated texts differ by their level of emotion.\u003c/p\u003e\n\u003cp\u003eAnother important task of text style transfer is sentiment transfer. However, it appears that the model performs poorly on the seq2seq sentiment transfer task (where the model almost always changes a minimal number of words). Nonetheless, when used in a conditional generation setting, we can see that the outputs are both accurate and consistent.\u003csup id=\"fnref:4\"\u003e\u003ca href=\"#fn:4\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e4\u003c/a\u003e\u003c/sup\u003e\u003c/p\u003e\n\u003cp\u003eNonetheless, from the examples above, one can also observe that in some instances, the generation accuracy is questionable, and occasionally, the generation fluency is not ensured. Notably, the model is prone to repetition. This is likely limited by the amount of training data (I only used less than 1/100 of the original dataset) and the insufficient training steps. Contrary to the original paper, I used the T5-base model instead of T5-large due to limited CUDA memory—this could potentially impair the power of the language model.\u003c/p\u003e\n\u003ch1 id=\"conclusion\"\u003e\n \u003cspan\u003eConclusion\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#conclusion\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h1\u003e\n\u003cp\u003eIn this project, attempts were made to implement a transformer-based text style transfer model. I tested multiple approaches and ultimately decided to implement a few-shot T5 style extractor model. Based on the vanilla T5 model provided by Hugging Face, I modified lower-level architectures and trained the model on preprocessed data in an unsupervised manner. Results demonstrate that the model is capable of transferring and generating texts with different styles, and its performance is validated on several tasks. Although the results are not as impressive as the state-of-the-art model, the model is remarkable in that it is trained on non-parallel data and can generalize to various tasks even when provided with only a few exemplars. This underscores the power of large, transformer-based pre-trained language models: with relatively minor modifications to the architecture and limited steps of pretraining, they can easily be generalized to diverse tasks.\u003c/p\u003e\n\u003ch1 id=\"style-exemplars-used\"\u003e\n \u003cspan\u003eStyle Exemplars Used\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#style-exemplars-used\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h1\u003e\n\u003cpre\u003e\u003ccode\u003eformal_exemplars = [\n \u0026quot;This was a remarkably thought-provoking read.\u0026quot;,\n \u0026quot;It is certainly amongst my favorites.\u0026quot;\n \u0026quot;We humbly request your presence at our gala on the 12th.\u0026quot;\n]\n\ninformal_exemplars = [\n \u0026quot;reading this rly makes u think\u0026quot;,\n \u0026quot;Its def one of my favs\u0026quot;,\n \u0026quot;come swing by our bbq next week if ya can make it\u0026quot;\n]\n\nreserved_exemplars = [ \n \u0026quot;No thank you, I'd prefer not to.\u0026quot;,\n \u0026quot;This game could have been better designed.\u0026quot;,\n \u0026quot;Do you know why they might have delayed the launch?\u0026quot;,\n \u0026quot;Sorry, I wasn' certain if you were joking.\u0026quot;\n]\n\nemotional_exemplars = [\n \u0026quot;Hell no, you can't make me do that.\u0026quot;,\n \u0026quot;This game is such a piece of garbage!\u0026quot;,\n \u0026quot;Why in god's name would they delay the damn launch? Are you frigging kidding me?\u0026quot;\n]\n\u003c/code\u003e\u003c/pre\u003e\n\u003ca class=\"action-link action-link--external\" href=\"https://github.com/xiyan128/text_style_transfer_transformer\" target=\"_blank\" rel=\"noopener noreferrer\"\u003eFull Code (GitHub)\u003c/a\u003e\n\n\u003cdiv class=\"footnotes\" role=\"doc-endnotes\"\u003e\n\u003chr\u003e\n\u003col\u003e\n\u003cli id=\"fn:1\"\u003e\n\u003cp\u003eColin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. \n\n\n\n\n\n\u003ca href=\"https://arxiv.org/abs/1910.10683\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003eExploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer\u003c/a\u003e\n.\u0026#160;\u003ca href=\"#fnref:1\" class=\"footnote-backref\" role=\"doc-backlink\"\u003e\u0026#x21a9;\u0026#xfe0e;\u003c/a\u003e\u003c/p\u003e\n\u003c/li\u003e\n\u003cli id=\"fn:2\"\u003e\n\u003cp\u003eNing Dai, Jianze Liang, Xipeng Qiu, and Xuanjing Huang. \n\n\n\n\n\n\u003ca href=\"https://aclanthology.org/P19-1601/\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003eStyle Transformer: Unpaired Text Style Transfer without Disentangled Latent Representation\u003c/a\u003e\n.\u0026#160;\u003ca href=\"#fnref:2\" class=\"footnote-backref\" role=\"doc-backlink\"\u003e\u0026#x21a9;\u0026#xfe0e;\u003c/a\u003e\u003c/p\u003e\n\u003c/li\u003e\n\u003cli id=\"fn:3\"\u003e\n\u003cp\u003eParker Riley, Noah Constant, Mandy Guo, Girish Kumar, David Bau, and Zak Stone. \n\n\n\n\n\n\u003ca href=\"https://arxiv.org/abs/2109.13370\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003eTextSETTR: Few-Shot, Tunable, and Interpretable Text Style Transfer\u003c/a\u003e\n.\u0026#160;\u003ca href=\"#fnref:3\" class=\"footnote-backref\" role=\"doc-backlink\"\u003e\u0026#x21a9;\u0026#xfe0e;\u003c/a\u003e\u003c/p\u003e\n\u003c/li\u003e\n\u003cli id=\"fn:4\"\u003e\n\u003cp\u003eFor exemplars of styles used in the few-shot inference, see the appendix. Positive and negative exemplars were sampled (n=100) from the Yelp review polarity dataset..\u0026#160;\u003ca href=\"#fnref:4\" class=\"footnote-backref\" role=\"doc-backlink\"\u003e\u0026#x21a9;\u0026#xfe0e;\u003c/a\u003e\u003c/p\u003e\n\u003c/li\u003e\n\u003c/ol\u003e\n\u003c/div\u003e\n","date_modified":"2026-06-10T00:40:55-07:00","date_published":"2022-03-15T23:15:33-07:00","id":"https://www.xiyan.dev/writings/2022/03/15/towards-unsupervised-few-shot-text-style-transfer-nlp-course-project/","summary":"An NLP course project on unsupervised few-shot text style transfer, building a T5 conditional-generation model with a style-extractor encoder, likely the first third-party implementation of TextSETTR-style targeted restyling.","tags":["NLP","Text Style Transfer"],"title":"Towards Unsupervised Few-Shot Text Style Transfer: NLP Course Project","url":"https://www.xiyan.dev/writings/2022/03/15/towards-unsupervised-few-shot-text-style-transfer-nlp-course-project/"},{"content_html":"\u003cblockquote\u003e\n\u003cp\u003ePersonal final notes for CSE 105 Theory of Computability. Not comprehensive\n\u0026ndash; only like a final exam cheat sheet.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003ch2 id=\"regular-expression\"\u003e\n \u003cspan\u003eRegular Expression\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#regular-expression\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003ePrecedence: $(R)$ → $R^{*}$ → $R_{1} R_{2}$ → $R_{1} \\cup R_{2}$\u003c/p\u003e\n\u003cp\u003e$\\emptyset$ represents empty set; $\\epsilon$ represents empty string; $R^+ \\equiv RR^*$\u003c/p\u003e\n\u003ch2 id=\"proving-closure\"\u003e\n \u003cspan\u003eProving closure\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#proving-closure\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003e\u003cstrong\u003eGiven\u003c/strong\u003e: What does it mean for L to be in class? \u003cem\u003ee.g.\u003c/em\u003e $L$* a regular language, so given a DFA/NFA* $M_L$\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eWTS\u003c/strong\u003e: The result of applying the operation to $L$ is still in this class.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConstruction\u003c/strong\u003e: Build a machine that recognizes the result of applying the operation to $L$. Start with description in English! \u003cem\u003ee.g. Let DFA/NFA\u003c/em\u003e $M=_\\cdots$* (w.r.t to* $M_L$\u003cem\u003e)\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCorrectness\u003c/strong\u003e: Prove $L(M) =$ result of applying operation to $L$\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\u003cem\u003eWTS 1\u003c/em\u003e: if $w$ is in set then $w$ is accepted by $M$\u003c/li\u003e\n\u003cli\u003e\u003cem\u003eWTS 2\u003c/em\u003e: if $w$ is not in the set then $w$ rejected by $M$.\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"prove-regularity\"\u003e\n \u003cspan\u003eProve regularity\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#prove-regularity\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003e\n\u003cp\u003eConstruction\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003eConstruct DFA;\u003c/li\u003e\n\u003cli\u003eConstruct NFA;\u003c/li\u003e\n\u003cli\u003eConstruct regular expression;\u003c/li\u003e\n\u003cli\u003eUse closure properties.\u003c/li\u003e\n\u003c/ol\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eProof of correctness\u003c/p\u003e\n\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003e\u003cstrong\u003eWTS 1\u003c/strong\u003e if w is in L then w is accepted by \u0026hellip;.; \u003cstrong\u003eWTS 2\u003c/strong\u003e if w is not in L then w is rejected by \u0026hellip;\u003c/p\u003e\n\u003ch2 id=\"to-show-a-language-l-is\"\u003e\n \u003cspan\u003eTo show a language L is\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#to-show-a-language-l-is\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003ch3 id=\"recognizable\"\u003e\n \u003cspan\u003eRecognizable\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#recognizable\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003eShow there is a TM M with L(M) = L.\u003c/li\u003e\n\u003cli\u003eUse closure properties.\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch3 id=\"not-recognizable\"\u003e\n \u003cspan\u003eNot recognizable\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#not-recognizable\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003eProve that L is not decidable and that the complement of L is recognizable.\u003c/li\u003e\n\u003cli\u003eUse closure properties.\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch3 id=\"decidable\"\u003e\n \u003cspan\u003eDecidable\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#decidable\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eShow there is a TM D that always halts and $L(D) = L$; Find a decidable\nproblem L\u0026rsquo; and show L reduces to L\u0026rsquo; Use closure properties.\u003c/p\u003e\n\u003ch3 id=\"not-decidable\"\u003e\n \u003cspan\u003eNot decidable\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#not-decidable\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eUse diagonalization Find an undecidable problem L\u0026rsquo; and show L\u0026rsquo; reduces to L. Use closure properties.\u003c/p\u003e\n\u003ctable\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth\u003e\u003c/th\u003e\n \u003cth\u003e\u003cstrong\u003eRegular Languages\u003c/strong\u003e\u003c/th\u003e\n \u003cth\u003e\u003cstrong\u003eCFL\u003c/strong\u003e\u003c/th\u003e\n \u003cth\u003e\u003cstrong\u003eDecidable Languages\u003c/strong\u003e\u003c/th\u003e\n \u003cth\u003e\u003cstrong\u003eRecognizable Languages\u003c/strong\u003e\u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd\u003eUnion\u003c/td\u003e\n \u003ctd\u003e✓\u003c/td\u003e\n \u003ctd\u003e✓\u003c/td\u003e\n \u003ctd\u003e✓\u003c/td\u003e\n \u003ctd\u003e✓\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003eIntersection\u003c/td\u003e\n \u003ctd\u003e✓\u003c/td\u003e\n \u003ctd\u003eX\u003c/td\u003e\n \u003ctd\u003e✓\u003c/td\u003e\n \u003ctd\u003e✓\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003eComplement\u003c/td\u003e\n \u003ctd\u003e✓\u003c/td\u003e\n \u003ctd\u003eX\u003c/td\u003e\n \u003ctd\u003e✓\u003c/td\u003e\n \u003ctd\u003eX\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003eStar\u003c/td\u003e\n \u003ctd\u003e✓\u003c/td\u003e\n \u003ctd\u003e✓\u003c/td\u003e\n \u003ctd\u003e✓\u003c/td\u003e\n \u003ctd\u003e✓\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003eConcatenation\u003c/td\u003e\n \u003ctd\u003e✓\u003c/td\u003e\n \u003ctd\u003e✓\u003c/td\u003e\n \u003ctd\u003e✓\u003c/td\u003e\n \u003ctd\u003e✓\u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003ch2 id=\"pumping-lemma\"\u003e\n \u003cspan\u003ePumping Lemma\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#pumping-lemma\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eIf $A$ is a regular language, then there is a number $p$ (the pumping length) where if $s$ is any string in $A$ of length at least $p$, then $s$ may be divided into three pieces, $s=x y z$, satisfying the following conditions:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003efor each $i \\geq 0, x y^{i} z \\in A$,\u003c/li\u003e\n\u003cli\u003e$|y|\u0026gt;0$, and\u003c/li\u003e\n\u003cli\u003e$|x y| \\leq p$.\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003e\u003cstrong\u003eProving strategies\u003c/strong\u003e\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eStructure\n\u003cul\u003e\n\u003cli\u003eAssume to the contrary that $L$ is regular\u003c/li\u003e\n\u003cli\u003eLet $p$ be the pumping length give by the pumping lemma\u003c/li\u003e\n\u003cli\u003eLet $s$ be … (w.r.t. $p$)\n\u003cul\u003e\n\u003cli\u003eBecause $s\\in L$ and $|s|\\ge p$, pumping lemma guarantees that $s=xyz$ satisfying the three condition of the lemma\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e[show that there’s no such valid division]\n\u003cul\u003e\n\u003cli\u003ecases depending on what $y$ contains\u003c/li\u003e\n\u003cli\u003econsider pump down\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eIf cannot prove directly, use closure properties\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch3 id=\"minimum-pumping-length\"\u003e\n \u003cspan\u003eMinimum pumping length\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#minimum-pumping-length\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003eFind the shortest string that can be pumped\u003c/li\u003e\n\u003cli\u003eIf has union\n\u003cul\u003e\n\u003cli\u003eOne side is described by the other side: ignore the union\u003c/li\u003e\n\u003cli\u003eNo: maximum of smallest pumping lengths of each individual language\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eIf can not be pumped: use the string length + 1\u003c/li\u003e\n\u003cli\u003eCheck if the lengths of strings have gaps\u003c/li\u003e\n\u003c/ul\u003e\n\u003ctable\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth\u003e\u003cstrong\u003eLanguage\u003c/strong\u003e\u003c/th\u003e\n \u003cth\u003e\u003cstrong\u003eDefinition\u003c/strong\u003e\u003c/th\u003e\n \u003cth\u003e\u003cstrong\u003eProperties\u003c/strong\u003e\u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd\u003e$A_\\text{DFA}$\u003c/td\u003e\n \u003ctd\u003e{⟨M,w⟩ | M is a DFA and M accepts w}\u003c/td\u003e\n \u003ctd\u003eDecidable\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e$A_\\text{TM}$\u003c/td\u003e\n \u003ctd\u003e{⟨M,w⟩ | M is a TM and M accepts w}\u003c/td\u003e\n \u003ctd\u003eUndecidable, RE, not CoRE\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e$E_\\text{DFA}$\u003c/td\u003e\n \u003ctd\u003e{⟨M⟩| M is a DFA and L(M)=∅}\u003c/td\u003e\n \u003ctd\u003eDecidable\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e$E_\\text{TM}$\u003c/td\u003e\n \u003ctd\u003e{⟨M⟩| M is a TM and L(M)=∅}\u003c/td\u003e\n \u003ctd\u003eUndecidable, not RE, CoRE\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e$EQ_\\text{DFA}$\u003c/td\u003e\n \u003ctd\u003e{⟨M1,M2⟩| M1 and M2 are DFAs and L(M1)=L(M2)}\u003c/td\u003e\n \u003ctd\u003eDecidable\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e$EQ_\\text{TM}$\u003c/td\u003e\n \u003ctd\u003e{⟨M1,M2⟩| M1 and M2 are TMs and L(M1)=L(M2)}\u003c/td\u003e\n \u003ctd\u003eUndecidable, not RE, not CoRE\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e$\\text{Diag}$\u003c/td\u003e\n \u003ctd\u003e{⟨M⟩ | M is a TM s.t. ⟨M⟩ is not in L(M)}\u003c/td\u003e\n \u003ctd\u003eUndecidable, not RE, CoRE\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e$\\text{HALT}_\\text{TM}$\u003c/td\u003e\n \u003ctd\u003e{⟨M,w⟩ | M is a TM and M halts on input w}\u003c/td\u003e\n \u003ctd\u003eUndecidable, RE, not CoRE\u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003ch2 id=\"mapping-reduction\"\u003e\n \u003cspan\u003eMapping Reduction\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#mapping-reduction\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eAssume $A\u0026lt;_m B$, i.e., there is a map reduction from $A$ to $B$. Then, we have\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eIf $B$ is RE, then $A$ is RE\u003c/li\u003e\n\u003cli\u003eIf $B$ is coRE, then $A$ is coRE\u003c/li\u003e\n\u003cli\u003eIf $B$ is decidable, then $A$ is decidable\u003c/li\u003e\n\u003cli\u003eIf $A$ is undecidable, then $B$ is undecidable\u003c/li\u003e\n\u003cli\u003eIf $A$ is not in RE, then $B$ is not in RE\u003c/li\u003e\n\u003cli\u003eIf $A$ is not in coRE, then $B$ is not in coRE\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eProof of correctness: Need to show two directions\u003c/p\u003e\n","date_modified":"2026-05-30T00:37:17-07:00","date_published":"2021-12-08T19:32:02-08:00","id":"https://www.xiyan.dev/writings/2021/12/08/theory-of-computability-notes/","summary":"Personal final-exam cheat sheet for CSE 105 Theory of Computability: regular languages, automata constructions, and the given / want-to-show / construction / correctness proof pattern. Not comprehensive.","tags":["Computer Science","Computability","Theory","Course"],"title":"Theory Of Computability Notes","url":"https://www.xiyan.dev/writings/2021/12/08/theory-of-computability-notes/"},{"content_html":"\u003ch2 id=\"differential-equations-classification\"\u003e\n \u003cspan\u003eDifferential Equations Classification\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#differential-equations-classification\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eOrdinary/Partial\n\u003cul\u003e\n\u003cli\u003einvolving only ordinary derivatives with respect to a single independent variable is called an ordinary differential equation.\u003c/li\u003e\n\u003cli\u003einvolving partial derivatives with respect to more than one independent variable is a partial differential equation.\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eLinear/Non-linear\n\u003cul\u003e\n\u003cli\u003eLinear: One in which the dependent variable $y$ and its derivatives appear in additive combinations of their first powers. May be written in the form\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003e$$\na_{n}(x) \\frac{d^{n} y}{d x^{n}}+a_{n-1}(x) \\frac{d^{n-1} y}{d x^{n-1}}+\\ldots+a_{1}(x) \\frac{d y}{d x}+a_{0}(x) y=F(x)\n$$\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eOrder\n\u003cul\u003e\n\u003cli\u003eThe order of the highest-order derivatives present in the equation.\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eHomogeneous/Non-homogeneous\n\u003cul\u003e\n\u003cli\u003eHomogeneous: One that does not have terms involving only $x$ (and constants).\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"explicitimplicit-solutions\"\u003e\n \u003cspan\u003eExplicit/Implicit Solutions\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#explicitimplicit-solutions\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003e\u003cstrong\u003eExplicit solution\u003c/strong\u003e is a function which when substituted in the equation for the depended variable satisfies the equation for any value of the independent variable within a given interval.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eImplicit solution\u003c/strong\u003e is a relation that defines one or more explicit solutions.\u003c/p\u003e\n\u003ch2 id=\"initial-value-problems\"\u003e\n \u003cspan\u003eInitial Value Problems\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#initial-value-problems\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003ch2 id=\"definition\"\u003e\n \u003cspan\u003eDefinition\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#definition\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eBy an Initial Value Problem (IVP) we mean the set of a $n$th order differential equation\u003c/p\u003e\n\u003cp\u003e$$\nF\\left(x, y, \\frac{d y}{d x}, \\ldots, \\frac{d^{n} y}{d x^{n}}\\right)=0\n$$\u003c/p\u003e\n\u003cp\u003eand the $n$ initial conditions\u003c/p\u003e\n\u003cp\u003e$$\ny\\left(x_{0}\\right)=y_{0}, \\frac{d y}{d x}\\left(x_{0}\\right)=y_{1}, \\ldots, \\frac{d^{n-1} y}{d x^{n-1}}\\left(x_{0}\\right)=y_{n-1}\n$$\u003c/p\u003e\n\u003cp\u003ewhere $y_{i}, i=0, \\ldots, n-1$ are given numbers.\u003c/p\u003e\n\u003ch2 id=\"solution-of-an-initial-value-problem\"\u003e\n \u003cspan\u003eSolution of an Initial Value Problem\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#solution-of-an-initial-value-problem\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eBy an explicit solution of the above defined Initial Value Problem we mean a function $y=y(x)$ such that satisfies the above differential equation and all the above initial conditions.\u003c/p\u003e\n\u003ch2 id=\"separable-equations\"\u003e\n \u003cspan\u003eSeparable Equations\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#separable-equations\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003ch3 id=\"determine-a-equation-is-separable\"\u003e\n \u003cspan\u003eDetermine a equation is separable\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#determine-a-equation-is-separable\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eIf the right-hand side of the equation $\\frac{d y}{d x}=f(x, y)$ can be expressed as a function $g(x)$ that depends only on $x$ times a function $p(y)$ that depends only on $y$, then the differential equation is called separable.\u003c/p\u003e\n\u003cp\u003eIn other words, a first-order equation is separable if it can be written in the following form.\u003c/p\u003e\n\u003cp\u003e$$\n\\frac{d y}{d x}=g(x) p(y)\n$$\u003c/p\u003e\n\u003ch3 id=\"solving-separable-equations\"\u003e\n \u003cspan\u003eSolving Separable Equations\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#solving-separable-equations\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003e$$\n\\frac{d y}{d x}=g(x) p(y) \\Longleftrightarrow \\int \\frac{1}{p(y)} d y=\\int g(x) d x\n$$\u003c/p\u003e\n\u003ch2 id=\"solving-linear-equations-with-an-integrating-factor\"\u003e\n \u003cspan\u003eSolving Linear Equations with an Integrating Factor\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#solving-linear-equations-with-an-integrating-factor\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eWe consider equations of the form\u003c/p\u003e\n\u003cp\u003e$$\n\\frac{d y}{d x}+P(x) y=Q(x)\n$$\u003c/p\u003e\n\u003cp\u003eThe key idea: multiply both sides with a function $\\mu(x)$ to get\u003c/p\u003e\n\u003cp\u003e$$\n\\mu(x) \\frac{d y}{d x}+\\mu(x) P(x) y=\\mu(x) Q(x)\n$$\u003c/p\u003e\n\u003cp\u003eand hope that we can determine $\\mu(x)$ such that we combine the terms $\\mu(x) \\frac{d y}{d x}$ and $r(x) P(x) y$ as\u003c/p\u003e\n\u003cp\u003e$$\n\\mu(x) \\frac{d y}{d x}+\\mu(x) P(x) y=\\frac{d}{d x}(\\mu(x) y)\n$$\u003c/p\u003e\n\u003cp\u003eSince the left hand side of the above equation is equal to the right hand side, the function $\\mu(x)$ has to satisfy the equation\u003c/p\u003e\n\u003cp\u003e$$\n\\mu^{\\prime}(x)=\\mu(x) P(x) \\rightarrow \\mu(x)=e^{\\int P(x) d x}\n$$\u003c/p\u003e\n\u003cp\u003eand thus we can write down the solution of the original equation (1) as\u003c/p\u003e\n\u003cp\u003e$$\ny(x)=\\frac{1}{\\mu(x)} \\int \\mu(x) Q(x) d x\n$$\u003c/p\u003e\n\u003ch2 id=\"check-exact-equation\"\u003e\n \u003cspan\u003eCheck Exact Equation\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#check-exact-equation\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003e$M(x, y) d x+N(x, y) d y=0$ is an exact equation iff\u003c/p\u003e\n\u003cp\u003e$$\n\\frac{\\partial M}{\\partial y}(x, y)=\\frac{\\partial N}{\\partial x}(x, y) \\quad \\forall x, y \\in \\mathbb{R} \\times \\mathbb{R}\n$$\u003c/p\u003e\n\u003ch2 id=\"exact-differential-equations-the-algorithm\"\u003e\n \u003cspan\u003eExact Differential Equations: The Algorithm\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#exact-differential-equations-the-algorithm\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eIf $M(x, y) d x+N(x, y) d y=0$ is exact then check exact equation.\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003e$\\frac{\\partial F}{\\partial x}=M \\Longrightarrow F(x, y)=\\int M(x, y) d x+g(y)$\u003c/li\u003e\n\u003cli\u003eTake the partial derivative with respect to $y$ of the above, make it equal to $N$ and solve for $g \\prime(y)$\u003c/li\u003e\n\u003cli\u003eIntegrate $g\\prime (y)$ to get $g(y)$ and substitute back to the above equation to get the solution $F(x, y)$\u003c/li\u003e\n\u003c/ol\u003e\n\u003ch2 id=\"special-integrating-factors\"\u003e\n \u003cspan\u003eSpecial Integrating Factors\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#special-integrating-factors\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003e$$\n\\begin{aligned}\n\u0026amp;\\frac{\\frac{\\partial M}{\\partial y}-\\frac{\\partial N}{\\partial x}}{N}=T(x) \\Longrightarrow \\mu(x, y)=\\mu(x)=\\exp \\left[\\int T(x) d x\\right] \\\n\u0026amp;\\frac{\\frac{\\partial N}{\\partial x}-\\frac{\\partial M}{\\partial y}}{M}=S(y) \\Longrightarrow \\mu(x, y)=\\mu(y)=\\exp \\left[\\int S(y) d y\\right]\n\\end{aligned}\n$$\u003c/p\u003e\n\u003ch2 id=\"solution-lost-or-gained\"\u003e\n \u003cspan\u003eSolution lost or gained\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#solution-lost-or-gained\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003eLost: When multiplying $\\frac {p(x, y)} {q(x, y)}$, check if $q(x, y)=0$ is a solution to the original equation\u003c/li\u003e\n\u003cli\u003eGained: When multiplying $\\mu(x, y)$, identify those solutions to $\\mu(x, y)=0$ that are not solutions to the original equation.\u003c/li\u003e\n\u003c/ol\u003e\n\u003ch2 id=\"solving-second-order-homogeneous-linear-equations\"\u003e\n \u003cspan\u003eSolving Second Order Homogeneous Linear Equations\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#solving-second-order-homogeneous-linear-equations\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eSolving\u003c/p\u003e\n\u003cp\u003e$$\nay{\\prime \\prime} + by{\\prime} + cy = 0\n$$\u003c/p\u003e\n\u003cp\u003eSolve auxiliary algebraic equation\u003c/p\u003e\n\u003cp\u003e$$\na r^{2}+r+c=0\n$$\u003c/p\u003e\n\u003cp\u003eThere are three cases\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003e$r_1 \\neq r_2$ → $y(x) = C_1 e^{r_1 x} + C_2 e^{r_2 x}$\u003c/li\u003e\n\u003cli\u003e$r_1 = r_2$ → $y(x)=C_{1} e^{\\lambda x}+C_{2} x e^{\\lambda x}$\u003c/li\u003e\n\u003cli\u003eComplex roots\n\u003cul\u003e\n\u003cli\u003eIf the auxiliary equation for the equation ay ${ }^{\\prime \\prime}+\\mathrm{by}^{\\prime}+\\mathrm{cy}=0$ has complex conjugate roots $\\alpha \\pm \\beta i$, then two linearly independent solutions to the equation are $e^{\\alpha t} \\cos \\beta \\mathrm{t}$ and $e^{\\alpha t} \\sin \\beta \\mathrm{t}$.\n\u003cul\u003e\n\u003cli\u003eA general solution can be expressed as a linear combination of these two solutions where $c_{1}$ and $c_{2}$ are arbitrary constants.\n$\ny(t)=c_{1} e^{\\alpha t} \\cos \\beta t+c_{2} e^{\\alpha t} \\sin \\beta t\n$\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003c/ol\u003e\n\u003ch2 id=\"theorem-wronskian--linear-independence\"\u003e\n \u003cspan\u003eTheorem (Wronskian \u0026amp; linear independence)\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#theorem-wronskian--linear-independence\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eThe Wronskian of $y_{1}(x), y_{2}(x)$ is defined to be\u003c/p\u003e\n\u003cp\u003e$$\nW\\left(y_{1}, y_{2}\\right):=y_{1} y_{2}^{\\prime}-y_{2} y_{1}^{\\prime}\n$$\u003c/p\u003e\n\u003cp\u003e$$y_{1}, y_{2}$ are linearly dependent on an interval I iff $W\\left(y_{1}, y_{2}\\right)(x)=0, \\forall x \\in I$.\u003c/p\u003e\n\u003ch2 id=\"undetermined-coefficients\"\u003e\n \u003cspan\u003eUndetermined coefficients\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#undetermined-coefficients\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eFirst, solve the homogeneous differential equation.\u003c/p\u003e\n\u003cp\u003eTo find a particular solution to the differential equation for $a\\prime\\prime +b y \\prime + cy = Ct^{m} e^{\\alpha t} \\cos \\beta t$ or $a\\prime\\prime +b y \\prime + cy = Ct^{m} e^{\\alpha t} \\sin \\beta t$, use the trial solution$y_{p}(t)=t^{s}\\left(A_{m} t^{m}+\\cdots+A_{1} t+A_{0}\\right) e^{\\alpha t} \\cos \\beta t+t^{s}\\left(B_{m} t^{m}+\\cdots+B_{1} t+B_{0}\\right) e^{\\alpha t} \\sin \\beta t$, with\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e$s=0$ if $\\alpha+i \\beta$ is not a root of the associated auxiliary equation and\u003c/li\u003e\n\u003cli\u003e$s=1$ if $\\alpha+i \\beta$ is a root of the associated auxiliary equation.\u003c/li\u003e\n\u003cli\u003e$s=2$ if $r$ is a double root of the associated auxiliary equation\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eDetermine the values for $m, \\alpha, \\beta$, and $s$ to create the trial solution. Substitute the trial solution into the given differential equation and determine the unknown coefficients $A_{j}$ by equating the coefficients of like terms.\u003c/p\u003e\n\u003ch2 id=\"variation-of-parameters\"\u003e\n \u003cspan\u003eVariation of parameters\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#variation-of-parameters\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eTo determine a particular solution to the non-homogeneous differential equation $ay\\prime\\prime+b y{\\prime}+c y=f$, take a particular solution of the non-homogeneous differential equation to be $y_{p}(t)=v_{1}(t) y_{1}(t)+v_{2}(t) y_{2}(t)$ for some functions $v_{1}(t)$ and $v_{2}(t)$.\u003c/p\u003e\n\u003cp\u003eNow determine $v_{1}(t)$ and $v_{2}(t)$ by solving the system below for $v_{1}{\\prime}(t)$ and $v_{2}{\\prime}(t)$ and integrating.\u003c/p\u003e\n\u003cp\u003e$$\n\\begin{aligned}\ny_{1} v_{1}^{\\prime}+y_{2} v_{2}^{\\prime} \u0026amp;=0 \\\ny_{1}^{\\prime} v_{1}^{\\prime}+y_{2}^{\\prime} v_{2}^{\\prime} \u0026amp;=\\frac{f}{a}\n\\end{aligned}\n$$\u003c/p\u003e\n\u003cp\u003eIntegrate to find $v_1$ and $v_2$. Or, the general formula for $v_{1}, v_{2}$ is\u003c/p\u003e\n\u003cp\u003e$$\nv_{1}=\\int \\frac{-f y_{2}}{a W\\left(y_{1}, y_{2}\\right)} d x, \\quad v_{2}=\\int \\frac{f{y_{1}}}{a W\\left(y_{1}, y_{2}\\right)} d x\n$$\u003c/p\u003e\n\u003ch2 id=\"method-for-solving-cauchy-euler-equations\"\u003e\n \u003cspan\u003eMethod for solving Cauchy Euler Equations\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#method-for-solving-cauchy-euler-equations\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eA linear second order equation that can be expressed in the form\u003c/p\u003e\n\u003cp\u003e$$\na x^{2} y^{\\prime \\prime}(x)+b x y^{\\prime}(x)+c y(x)=f(x)\n$$\u003c/p\u003e\n\u003cp\u003ewhere $a, b, c$ are constants, is called a Cauchy-Euler, or equidimensional equation.\u003c/p\u003e\n\u003cp\u003eSubstitute $y=x^{r}$ to get the auxiliary equation\u003c/p\u003e\n\u003cp\u003e$$\na r^{2}+(b-a) r+c=0\n$$\u003c/p\u003e\n\u003cp\u003eIf this equation has\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003etwo real roots $r_{1} \\neq r_{2}$ then $x^{r_{1}}, x^{r_{2}}$ are two solutions.\u003c/li\u003e\n\u003cli\u003eone repeated real root $r_{1}=r_{2}=r$ then $x^{r}, x^{r} \\ln x$ are two solutions.\u003c/li\u003e\n\u003cli\u003etwo complex roots $\\alpha+\\beta i$ then $x^{\\alpha} \\cos (\\beta \\ln x), x^{\\alpha} \\sin (\\beta \\ln x)$ are two solutions.\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"linear-algebra\"\u003e\n \u003cspan\u003eLinear Algebra\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#linear-algebra\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eSingular Matrix → $\\det=0$, no inverse\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003eCompute $\\lambda$ and $v$ such that $A v=\\lambda v \\Rightarrow(A-\\lambda I) v=0$.\u003c/li\u003e\n\u003cli\u003eFor nontrivial solutions we need to have $|A-\\lambda I|=0$. Solve for $\\lambda$.\u003c/li\u003e\n\u003cli\u003ePlug-in the computed value of the eigenvalue $\\lambda$ in $(A-\\lambda I) \\mathrm{v}=0$ to compute, associated to $\\lambda$, eigenvector $v$.\u003c/li\u003e\n\u003c/ol\u003e\n\u003ch2 id=\"wronskian\"\u003e\n \u003cspan\u003eWronskian\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#wronskian\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eThe Wronskian of two differentiable functions $f$ and $g$ is $W(f, g)=f g^{\\prime}-g f^{\\prime}$.\u003c/p\u003e\n\u003cp\u003eMore generally, for $n$ real- or complex-valued functions $f_{1}, \\ldots, f_{n}$, which are $n-1$ times differentiable on an interval $I$, the Wronskian $W\\left(f_{1}, \\ldots, f_{n}\\right)$ as a function on $I$ is defined by\u003c/p\u003e\n\u003cp\u003eFor vector valued functions, the Wronskian is just the determinant of the matrix where each column is a function.\u003c/p\u003e\n\u003cp\u003eOn an interval $I$ where the entries of $A(t)$ are continuous, let $x_1$ and $x_{2}$ be two solutions and $W(t)$ their Wronskian. Then either\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e$W(t) \\equiv 0$ on $I$, and $x_{1}$ and $x_{2}$ are linearly dependent on $I$, or\u003c/li\u003e\n\u003cli\u003e$W(t)$ is never 0 on $I$, and $x_{1}$ and $x_{2}$ are linearly independent on $I$.\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch1 id=\"solving-xprimetmathbf-axt\"\u003e\n \u003cspan\u003eSolving $x^\\prime(t)=\\mathbf Ax(t)$\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#solving-xprimetmathbf-axt\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h1\u003e\n\u003cp\u003eSuppose the $n \\times n$ matrix $\\mathbf A$ has $n$ linearly independent eigenvectors $\\mathbf{u_1, u_2, \\ldots u_n}$. Let $r_i$ be the eigenvalue corresponding to $\\mathbf u_i$. Then ${\\mathbf{e^{r_1t} u_1, e^{r_2t} u_2, \\ldots e^{r_nt} u_n}}$is a fundamental solution set. Consequently, a general solution is $\\mathbf x(t) = \\mathbf{e^{r_1t} u_1 + e^{r_2t} u_2 + \\ldots + e^{r_nt}u_n}$ where $c_1, c_2, \\ldots, c_n$ are arbitrary constants.\u003c/p\u003e\n\u003ch2 id=\"same-eigenvalues-lambda_1--lambda_2--ldots--lambda_n\"\u003e\n \u003cspan\u003eSame eigenvalues $\\lambda_1 = \\lambda_2 = \\ldots = \\lambda_n$\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#same-eigenvalues-lambda_1--lambda_2--ldots--lambda_n\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eTry finding two linearly independent eigenvectors $\\vec v_1, \\vec v_2, \\ldots \\vec v_n$\n\u003cul\u003e\n\u003cli\u003eIf found, do the same as above\u003c/li\u003e\n\u003cli\u003eIf not, do this\n\u003col\u003e\n\u003cli\u003eStart with eigenvalue $\\vec v_1$ (solution to $(\\mathbf A - \\lambda \\mathbf I)\\vec v=0$)\u003c/li\u003e\n\u003cli\u003eFind $\\vec v_2$ from $(\\mathbf A - \\lambda \\mathbf I)\\vec v=\\vec v_1$\u003c/li\u003e\n\u003cli\u003eRepeat to find Find $\\vec v_n$ from $(\\mathbf A - \\lambda \\mathbf I)\\vec v=\\vec v_{n-1}$\u003c/li\u003e\n\u003cli\u003eSolution will be:\u003c/li\u003e\n\u003c/ol\u003e\n\u003cul\u003e\n\u003cli\u003egeneral solution to the homo.\u003c/li\u003e\n\u003cli\u003e$\\vec x_1 = \\vec v_1 e^{\\lambda t}$\u003c/li\u003e\n\u003cli\u003e$\\vec x_2= t \\vec x_1\\ + \\vec v_2 e^{\\lambda t}$ (note that $\\vec x_1$ is from the previous line)\u003c/li\u003e\n\u003cli\u003e…\u003c/li\u003e\n\u003cli\u003e$\\vec{x_{n}}=e^{\\lambda t}\\left(\\vec{v_{n}}+t v_{n-1}+\\cdots+\\frac{t^{n-1}}{(n-1) !} \\vec{v_{1}}\\right)$\u003c/li\u003e\n\u003cli\u003eSuperposition of the above\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"complex-eigenvalues\"\u003e\n \u003cspan\u003eComplex Eigenvalues\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#complex-eigenvalues\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eIf the real matrix $A$ has complex conjugate eigenvalues $\\alpha \\pm i \\beta$ with corresponding eigenvectors $\\mathrm{a} \\pm i \\mathrm{~b}$, then two linearly independent. To find the complex eigenvectors, use complex conjugate in ref.\nComplex vector solutions to $x^{\\prime}(t)=Ax(t)$ are\u003c/p\u003e\n\u003cp\u003e$$\n\\mathrm{x}\u003cem\u003e{1}=(\\mathrm{a}+i \\mathrm{~b}) e^{(\\alpha+i \\beta) t}, \\quad \\mathrm{x}\u003c/em\u003e{2}=(\\mathrm{a}-i \\mathrm{~b}) e^{(\\alpha-i \\beta) t}\n$$\u003c/p\u003e\n\u003cp\u003eWr equivalently to the following two real vector solutions\u003c/p\u003e\n\u003cp\u003e$$\n\\mathrm{x}\u003cem\u003e{1}=e^{\\alpha t} \\cos \\beta t \\mathrm{a}-e^{\\alpha t} \\sin \\beta t \\mathrm{~b}, \\quad \\mathrm{x}\u003c/em\u003e{2}=e^{\\alpha t} \\sin \\beta t \\mathrm{a}+e^{\\alpha t} \\cos \\beta t \\mathrm{~b}\n$$\u003c/p\u003e\n\u003ch1 id=\"solving-xprimetmathbf-axt--vec-ft\"\u003e\n \u003cspan\u003eSolving $x^\\prime(t)=\\mathbf Ax(t) + \\vec f(t)$\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#solving-xprimetmathbf-axt--vec-ft\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h1\u003e\n\u003cul\u003e\n\u003cli\u003eLet $\\vec x_g = c_1 \\vec x_1 + c_2 \\vec x_2 + \\ldots + c_n \\vec x_n$ be solution to the homogeneous system of equations; let $\\vec x_p$ be a particular solution to the non homogeneous system of equations.\u003c/li\u003e\n\u003c/ul\u003e\n\u003ctable\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth\u003eIf $\\vec f$ contains\u003c/th\u003e\n \u003cth\u003eGuess\u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd\u003e$t^k (k \\in \\mathbb Z^+)$\u003c/td\u003e\n \u003ctd\u003e$\\vec a_{k} t^{k} + \\vec a_{k-1} t^{k-1} + \\ldots + \\vec a_1 + \\vec a_0$\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e$\\sin t$\u003c/td\u003e\n \u003ctd\u003e$\\vec a_1 \\sin t+\\vec a_2 \\cos t$\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e$e^{\\lambda t}$ (not solution to homo.)\u003c/td\u003e\n \u003ctd\u003e$\\vec a_1 e^{\\lambda t}$\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e$e^{\\lambda t}$ (solution to homo.)\u003c/td\u003e\n \u003ctd\u003e$(\\vec a_{n} t^{n} + \\vec a_{n-1} t^{n-1} + \\ldots + \\vec a_1 + \\vec a_0) e^{\\lambda t}$\u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003ch2 id=\"the-matrix-exponential-function\"\u003e\n \u003cspan\u003eThe Matrix Exponential Function\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#the-matrix-exponential-function\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003e$\\vec x \\prime = \\mathbf P \\vec x$ has solution $\\vec x = e^{t\\mathbf P} \\vec c$.\u003c/p\u003e\n\u003cp\u003eTo calculate $e^{t \\mathbf P}$:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eIf $\\mathbf P$ is diagonal, $\\begin{pmatrix} \\lambda_1 \u0026amp; \\cdots \u0026amp; 0 \\ \\vdots \u0026amp; \\ddots \u0026amp; \\vdots \\ 0 \u0026amp; \\cdots \u0026amp; \\lambda_n \\end{pmatrix}$. $e^{t \\mathbf P}=\\begin{pmatrix} e^{t \\lambda_1} \u0026amp; \\cdots \u0026amp; 0 \\ \\vdots \u0026amp; \\ddots \u0026amp; \\vdots \\ 0 \u0026amp; \\cdots \u0026amp; e^{t \\lambda_n} \\end{pmatrix}$\u003c/li\u003e\n\u003cli\u003eOtherwise, let $(\\lambda_1, \\vec v_1), \\cdots, (\\lambda_n, \\vec v_n)$ be Eigenpairs of $\\mathbf P$\n\u003cul\u003e\n\u003cli\u003e$e^{t\\mathbf P}=\\mathbf E e^{t \\mathbf D} \\mathbf E^{-1}$\u003c/li\u003e\n\u003cli\u003e$\\mathbf E = \\begin{pmatrix} | \u0026amp; \u0026amp; | \\ \\vec v_1 \u0026amp; \\cdots \u0026amp; \\vec v_n \\ | \u0026amp; \u0026amp; | \\end {pmatrix}$, $\\mathbf D = \\begin{pmatrix} \\lambda_1 \u0026amp; \\cdots \u0026amp; 0 \\ \\vdots \u0026amp; \\ddots \u0026amp; \\vdots \\ 0 \u0026amp; \\cdots \u0026amp; \\lambda_n \\end{pmatrix}$\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eNon-diagonalizable\n\u003cul\u003e\n\u003cli\u003eNot enough linearly independent eigenvectors\u003c/li\u003e\n\u003cli\u003eCannot be written as $\\mathbf E \\mathbf D \\mathbf E^{-1}$ for some diagonal matrix $\\mathbf D$\u003c/li\u003e\n\u003cli\u003eUse \u0026ldquo;Same eigenvalues\u0026rdquo;\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch1 id=\"laplace-transform\"\u003e\n \u003cspan\u003eLaplace Transform\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#laplace-transform\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h1\u003e\n\u003ca class=\"action-link action-link--download\" href=\"https://res.craft.do/user/full/66475e26-27ac-fc94-573f-cae0968e70be/doc/00E1E467-3A0E-49B4-8639-58A34095E645/08A5AA83-2CA2-4264-B63D-7BCF2D9AB532_2/Properties%20of%20Laplace%20Transforms.pdf\"\u003eProperties of Laplace Transforms\u003c/a\u003e\n\n\u003ca class=\"action-link action-link--download\" href=\"https://res.craft.do/user/full/66475e26-27ac-fc94-573f-cae0968e70be/doc/00E1E467-3A0E-49B4-8639-58A34095E645/E5752AF3-72FB-4A8A-9D20-B144D0FD8BCF_2/Table%20of%20Laplace%20Transforms.pdf\"\u003eTable of Laplace Transforms\u003c/a\u003e\n\n\u003cp\u003eDefinition: $\\mathcal{L}{f(t)}=F(s)=\\int_{0}^{\\infty} e^{-s t} f(t) d t$\u003c/p\u003e\n\u003ctable\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth\u003e$f(t)$\u003c/th\u003e\n \u003cth\u003e$F(t)$\u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd\u003e$C$\u003c/td\u003e\n \u003ctd\u003e$\\frac C s$\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e$t$\u003c/td\u003e\n \u003ctd\u003e$\\frac 1 {s^2}$\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e$t^2$\u003c/td\u003e\n \u003ctd\u003e$\\frac 2 {s^3}$\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e$t^n$\u003c/td\u003e\n \u003ctd\u003e$\\frac {n!} {s^{n+1}}$\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e$e^{-at}$\u003c/td\u003e\n \u003ctd\u003e$\\frac 1 {s+a}$\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e$\\sin \\omega t$\u003c/td\u003e\n \u003ctd\u003e$\\frac{\\omega}{s^{2}+\\omega^{2}}$\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e$\\cos \\omega t$\u003c/td\u003e\n \u003ctd\u003e$\\frac{s}{s^{2}+\\omega^{2}}$\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e$\\sinh \\omega t$\u003c/td\u003e\n \u003ctd\u003e$\\frac{\\omega}{s^{2}-\\omega^{2}}$\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e$\\cosh \\omega t$\u003c/td\u003e\n \u003ctd\u003e$\\frac{s}{s^{2}-\\omega^{2}}$\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e$u(t-a)$\u003c/td\u003e\n \u003ctd\u003e$\\frac{e^{-a s}}{s}$\u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003ctable\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth\u003e$g^{\\prime}(t)$\u003c/th\u003e\n \u003cth\u003e$s G(s)-g(0)$\u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd\u003e$g^{\\prime \\prime}(t)$\u003c/td\u003e\n \u003ctd\u003e$s^{2} G(s)-s g(0)-g^{\\prime}(0)$\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e$g^{\\prime \\prime \\prime}(t)$\u003c/td\u003e\n \u003ctd\u003e$s^{3} G(s)-s^{2} g(0)-s g^{\\prime}(0)-g^{\\prime \\prime}(0)$\u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003ch2 id=\"solving-de-with-laplace-transforms\"\u003e\n \u003cspan\u003eSolving DE with Laplace Transforms\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#solving-de-with-laplace-transforms\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003eTake the Laplace transform of both sides of the equation.\u003c/li\u003e\n\u003cli\u003eUse the properties of the Laplace transform and the initial conditions to obtain an equation for the Laplace transform of the solution and then solve this equation for the transform.\u003c/li\u003e\n\u003cli\u003eDetermine the inverse Laplace transform of the solution by looking it up in a table or by using a suitable method (such as partial fractions) in combination with the table.\u003c/li\u003e\n\u003c/ol\u003e\n","date_modified":"2026-05-30T00:37:17-07:00","date_published":"2021-12-04T23:48:15-08:00","id":"https://www.xiyan.dev/writings/2021/12/04/introduction-to-differential-equations-notes/","summary":"Personal notes for Math 20D Introduction to Differential Equations: classifying ODEs and PDEs, linearity and order, and the core techniques for solving them.","tags":["Differential Equations"],"title":"Introduction to Differential Equations Notes","url":"https://www.xiyan.dev/writings/2021/12/04/introduction-to-differential-equations-notes/"},{"content_html":"\u003cblockquote\u003e\n\u003cp\u003ePersonal notes for CSE20 - Discrete Mathematics for Computer Science. Taken Fall 2020, with professor Shachar Lovett. See \n\n\n\n\n\n\u003ca href=\"https://cseweb.ucsd.edu/classes/fa20/cse20-a/\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003eCourse Homepage\u003c/a\u003e\n\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003ch1 id=\"sets\"\u003e\n \u003cspan\u003eSets\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#sets\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h1\u003e\n\u003ch2 id=\"recursive-definition-of-sets\"\u003e\n \u003cspan\u003eRecursive definition of sets\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#recursive-definition-of-sets\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003e\u003cstrong\u003eBasis step:\u003c/strong\u003e Specify finitely many elements of $S$\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eRecursive step:\u003c/strong\u003e Give a rule for creating a new element of $S$ from known values existing in $S$, and potentially other values.\u003c/p\u003e\n\u003ch3 id=\"string\"\u003e\n \u003cspan\u003eString\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#string\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eThe set $\\Sigma^*$of strings over the alphabet $\\Sigma$ is defined recursively by\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eBasis step:\u003c/strong\u003e $\\lambda \\in \\Sigma^*$ (where $\\lambda$ is the empty string containing no symbols)\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eRecursive step:\u003c/strong\u003e $w\\in \\Sigma^* \\land x\\in\\Sigma \\rightarrow wx \\in \\Sigma^*$.\u003c/p\u003e\n\u003ch2 id=\"set-operations\"\u003e\n \u003cspan\u003eSet operations\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#set-operations\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003ch3 id=\"set-equality\"\u003e\n \u003cspan\u003eSet equality\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#set-equality\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eWhen $A$ and $B$ are sets, $A = B$ means $∀x(x ∈ A\\leftrightarrow x ∈ B)$\u003c/p\u003e\n\u003ch3 id=\"subset\"\u003e\n \u003cspan\u003eSubset\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#subset\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eWhen $A$ and $B$ are sets, $A ⊆ B$ means $∀x(x ∈ A → x ∈ B)$\u003c/p\u003e\n\u003ch3 id=\"proper-subset\"\u003e\n \u003cspan\u003eProper subset\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#proper-subset\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eWhen $A$ and $B$ are sets, $A\\subsetneq B$\nmeans $(A ⊆ B) ∧ (A = B)$\u003c/p\u003e\n\u003ch3 id=\"union\"\u003e\n \u003cspan\u003eUnion\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#union\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eWhen $A$ and $B$ are sets, $A ∪ B = {x | x ∈ A ∨ x ∈ B}$\u003c/p\u003e\n\u003ch3 id=\"intersection\"\u003e\n \u003cspan\u003eIntersection\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#intersection\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eWhen $A$ and $B$ are sets, $A ∩ B = {x | x ∈ A ∧ x ∈ B}$\u003c/p\u003e\n\u003ch3 id=\"set-difference\"\u003e\n \u003cspan\u003eSet difference\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#set-difference\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eWhen $A$ and $B$ are sets, $A - B = {x | x ∈ A ∧ x \\not∈ B}$\u003c/p\u003e\n\u003ch3 id=\"disjoint-sets\"\u003e\n \u003cspan\u003eDisjoint sets\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#disjoint-sets\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eSets $A$ and $B$ are disjoint means $A ∩ B = ∅$\u003c/p\u003e\n\u003ch3 id=\"power-set\"\u003e\n \u003cspan\u003ePower set\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#power-set\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eWhen $S$ is a set, $P (S) = {X | X ⊆ S}$\u003c/p\u003e\n\u003ch3 id=\"cartesian-product\"\u003e\n \u003cspan\u003eCartesian product\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#cartesian-product\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003e$A \\times B={(a, b) \\mid a \\in A \\text { and } b \\in B}$\u003c/p\u003e\n\u003ch3 id=\"set-wise-concatenation\"\u003e\n \u003cspan\u003eSet-wise concatenation\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#set-wise-concatenation\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003e$A \\circ B={a b \\mid a \\in A \\text { and } b \\in B}$\u003c/p\u003e\n\u003ch2 id=\"sets-of-integers\"\u003e\n \u003cspan\u003eSets of Integers\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#sets-of-integers\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003e$\\mathbb N$ ⇒ The set of natural numbers, ${1,2,3,\\cdots}$\u003c/p\u003e\n\u003cp\u003e$\\mathbb Z$ ⇒ The set of integers, ${\\cdots,-2,-1,0,1,2,\\cdots}$\u003c/p\u003e\n\u003cp\u003e$\\mathbb Z^+$⇒ The set of positive integers ${1,2,3,\\cdots}$\u003c/p\u003e\n\u003cp\u003e$\\mathbb Z^{\\neq 0}$⇒ The set of nonzero integers\u003c/p\u003e\n\u003ch2 id=\"primes-and-rationals\"\u003e\n \u003cspan\u003ePrimes and rationals\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#primes-and-rationals\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eWhen $a$ and $b$ are integers and $a$ is nonzero, $a$ \u003cstrong\u003edivides\u003c/strong\u003e $b$ means there is an integer $c$ such that $b = ac$.\u003c/p\u003e\n\u003cp\u003eSymbolically, $F(a, b)=\\exists c \\in \\mathbb{Z}(b=a c)$ and is a predicate over the domain $\\mathbb{Z}^{\\neq 0} \\times \\mathbb{Z}$.\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eTerminology\u003c/em\u003e: $a$ is a \u003cstrong\u003efactor\u003c/strong\u003e of $b$, $a$ is a \u003cstrong\u003edivisor\u003c/strong\u003e of $b$, $b$ is a \u003cstrong\u003emultiple\u003c/strong\u003e of $a$, $a|b$.\u003c/p\u003e\n\u003ch3 id=\"prime\"\u003e\n \u003cspan\u003ePrime\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#prime\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eAn integer $p$ greater than $1$ is called \u003cstrong\u003eprime\u003c/strong\u003e means the only positive factors of $p$ are $1$ and $p$. A positive integer that is greater than $1$ and is not prime is called composite.\u003c/p\u003e\n\u003cp\u003eA formal definition of the predicate $Pr$ over the domain $\\mathbb Z$ which evaluates to $T$ exactly when the input is prime is\u003c/p\u003e\n\u003cp\u003e$$(x \u0026gt; 1) ∧ ∀a( ( a \u0026gt; 0 ∧ F (a, x) ) → (a = 1 ∨ a = x) )$$\u003c/p\u003e\n\u003ch3 id=\"rationals\"\u003e\n \u003cspan\u003eRationals\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#rationals\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003e$\\mathbb Q$ ⇒ $({\\frac{p}{q} \\mid p \\in \\mathbb{Z} \\text { and } q \\in \\mathbb{Z} \\text { and } q \\neq 0 )}$, or $({x \\in \\mathbb{R} \\mid \\exists p \\in \\mathbb{Z} \\exists q \\in \\mathbb{Z}^{+}(p=x \\cdot q))}$\u003c/p\u003e\n\u003ch3 id=\"division-algorithm\"\u003e\n \u003cspan\u003eDivision algorithm\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#division-algorithm\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eLet $n$ be an integer and $d$ a positive integer. There are unique integers $q$ and $r$, with $0 ≤ r \u0026lt; d$, such that $n = dq + r$. In this case, $d$ is called the divisor, $n$ is called the dividend, $q$ is called the quotient, and $r$ is called the remainder. We write $q = n \\text{ div } d$ and $r = n \\mod d$.\u003c/p\u003e\n\u003ch3 id=\"greatest-common-divisor\"\u003e\n \u003cspan\u003eGreatest common divisor\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#greatest-common-divisor\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eLet $a$ and $b$ be integers, not both zero. The largest integer $d$ such that $d$ is a factor of $a$ and $d$ is a factor of $b$ is called the greatest common divisor of $a$ and $b$ and is denoted by $\\gcd(a, b)$.\u003c/p\u003e\n\u003ch1 id=\"number-representations\"\u003e\n \u003cspan\u003eNumber representations\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#number-representations\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h1\u003e\n\u003ch2 id=\"base-expansion-and-conversions\"\u003e\n \u003cspan\u003eBase expansion and conversions\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#base-expansion-and-conversions\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eFor $b$ an integer greater than 1 and $n$ a positive integer, the \u003cstrong\u003ebase $b$ expansion\u003c/strong\u003e of $n$ is\u003c/p\u003e\n\u003cp\u003e$$((a_{k-1} \\cdots a_{1} a_{0}))_{b}$$\u003c/p\u003e\n\u003cp\u003ewhere \u003cem\u003ek\u003c/em\u003e is a positive integer, $a_{0}, a_{1}, \\ldots, a_{k-1}$ are\nnon negative integers less than $b, a_{k-1} \\neq 0,$ and\u003c/p\u003e\n\u003cp\u003e$$n=a_{k-1} b^{k-1}+\\cdots+a_{1} b+a_{0}$$\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eBase $b$ fixed-width $w$ expansion of $n$ ⇒\u003c/strong\u003e could have leading zeros.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eBase $b$ fixed-width expansion of $x$ with with integer part width $x$ and fractional part width $w\u0026rsquo;$⇒\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e$$x \\geq a_{w-1} b^{w-1}+\\cdots+a_{1} b+a_{0}+c_{1} b^{-1}+\\cdots+c_{w^{\\prime}} b^{-w^{\\prime}}$$\u003c/p\u003e\n\u003cp\u003eand\u003c/p\u003e\n\u003cp\u003e$$x\u0026lt;a_{w-1} b^{w-1}+\\cdots+a_{1} b+a_{0}+c_{1} b^{-1}+\\cdots+((c_{w^{\\prime}}+1)) b^{-w^{\\prime}}$$\u003c/p\u003e\n\u003ch2 id=\"representing-negative-integers\"\u003e\n \u003cspan\u003eRepresenting negative integers\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#representing-negative-integers\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003e\u003cstrong\u003eSigned-magnitude\u003c/strong\u003e ⇒$([1 a_{w-2} \\cdots a_{0})]\u003cem\u003e{s, w}$ where $n=((a\u003c/em\u003e{w-2} \\cdots a_{0}))_{2, w-1}$\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTwo\u0026rsquo;s complement\u003c/strong\u003e ⇒ $([1 a_{w-2} \\cdots a_{0})]\u003cem\u003e{2 c, w}$where $2^{w-1}-n=((a\u003c/em\u003e{w-2} \\cdots a_{0}))_{2, w-1}$\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTo represent $–n$ in 2s complement with width $w$\u003c/strong\u003e\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eCalculate $2^{w-1}-n$, convert to binary fixed-width $w-1$, pad with leading $1$\u003c/li\u003e\n\u003cli\u003eExpress $–n$ as a sum of powers of 2, where leftmost $(2w-1)$ is negative weight\u003c/li\u003e\n\u003cli\u003eConvert $n$ to fixed-width $w$ binary, flip bits, add $1$ (ignore overflow)\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch1 id=\"logics\"\u003e\n \u003cspan\u003eLogics\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#logics\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h1\u003e\n\u003ch2 id=\"logic-gates-and-circuits\"\u003e\n \u003cspan\u003eLogic gates and circuits\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#logic-gates-and-circuits\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003e$$\\begin{array}{cc||c|c|c}\u0026amp;\u0026amp; \\text { Conjunction } \u0026amp; \\text { Exclusive or } \u0026amp; \\text { Disjunction } \\\\p \u0026amp; q \u0026amp; p \\wedge q \u0026amp; p \\oplus q \u0026amp; p \\vee q \\\\\\hline T \u0026amp; T \u0026amp;T\u0026amp; F \u0026amp; T \\\\T \u0026amp; F \u0026amp; F \u0026amp; T \u0026amp; T \\\\F \u0026amp; T \u0026amp; F \u0026amp; T \u0026amp; T \\\\F \u0026amp; F \u0026amp; F \u0026amp; F \u0026amp; F\\end{array}$$\u003c/p\u003e\n\u003cp\u003e$$\\begin{array}{c||c}\\text { Input } \u0026amp; \\text { Output } \\\\ \u0026amp; \\text { Negation } \\\\ p \u0026amp; \\neg p \\\\ \\hline T \u0026amp; F \\\\ F \u0026amp; T\\end{array}$$\u003c/p\u003e\n\u003ch2 id=\"compound-proposition\"\u003e\n \u003cspan\u003eCompound proposition\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#compound-proposition\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003ch3 id=\"terminologies\"\u003e\n \u003cspan\u003eTerminologies\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#terminologies\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003e\u003cstrong\u003eLogically equivalent ⇒\u003c/strong\u003e same truth values for all settigns of truth values to their propostition variables\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTautology ⇒\u003c/strong\u003e compound proposition that evaluates to true for all settings of truth values to its propositional vaiables; it is abbreviated T\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eContradiction ⇒\u003c/strong\u003e compound proposition that evaluates to false for all settings of truth values to its propositional variables; it is abbreviated F\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConsistency ⇒\u003c/strong\u003e a collection of compound propositions is \u003cstrong\u003econsistent\u003c/strong\u003e means there is an assignment of truth values to the variables that makes all these propositions true.\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eNote:\u003c/em\u003e $p\\equiv q$ is NOT a compound proposition\u003c/p\u003e\n\u003ch3 id=\"cnf\"\u003e\n \u003cspan\u003eCNF\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#cnf\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eA compound proposition is in \u003cstrong\u003econjunctive normal form (CNF)\u003c/strong\u003e means that it is an AND of ORs of variables and their negations.\u003c/p\u003e\n\u003cp\u003eTo write ⇒ Look for rows that are false.\u003c/p\u003e\n\u003ch3 id=\"dnf\"\u003e\n \u003cspan\u003eDNF\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#dnf\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eA compound proposition is in \u003cstrong\u003edisjunctive normal form (DNF)\u003c/strong\u003e means that it is an OR of ANDs of variables and their negations.\u003c/p\u003e\n\u003cp\u003eTo write ⇒ Look for rows that are true\u003c/p\u003e\n\u003ch2 id=\"conditionals\"\u003e\n \u003cspan\u003eConditionals\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#conditionals\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003ch3 id=\"terminologies-1\"\u003e\n \u003cspan\u003eTerminologies\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#terminologies-1\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eThe \u003cstrong\u003ehypothesis\u003c/strong\u003e of $p → q$ is $p$\u003c/p\u003e\n\u003cp\u003eThe \u003cstrong\u003eantecedent\u003c/strong\u003e of $p → q$ is $p$\u003c/p\u003e\n\u003cp\u003eThe \u003cstrong\u003econclusion\u003c/strong\u003e of $p → q$ is $q$\u003c/p\u003e\n\u003cp\u003eThe \u003cstrong\u003econsequent\u003c/strong\u003e of $p → q$ is $q$\u003c/p\u003e\n\u003cp\u003eThe \u003cstrong\u003econverse\u003c/strong\u003e of $p \\rightarrow q$ is $q \\rightarrow p$\u003c/p\u003e\n\u003cp\u003eThe \u003cstrong\u003einverse\u003c/strong\u003e of p $\\rightarrow q$ is $\\neg p \\rightarrow \\neg q$\u003c/p\u003e\n\u003cp\u003eThe \u003cstrong\u003econtrapositive\u003c/strong\u003e of $p \\rightarrow q$ is $\\neg q \\rightarrow \\neg p$\u003c/p\u003e\n\u003ch3 id=\"truth-table\"\u003e\n \u003cspan\u003eTruth table\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#truth-table\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003e$$\\begin{array}{cc||c|c|c|c|c} \u0026amp; \u0026amp; \\text { AND } \u0026amp; \\text { XOR } \u0026amp; \\text { OR } \u0026amp; \\text { Conditional } \u0026amp; \\text { Biconditional } \\\\ p \u0026amp; q \u0026amp; p \\wedge q \u0026amp; p \\oplus q \u0026amp; p \\vee q \u0026amp; p \\rightarrow q \u0026amp; p \\leftrightarrow q \\\\ \\hline T \u0026amp; T \u0026amp; T \u0026amp; F \u0026amp; T \u0026amp; T \u0026amp; T \\\\ T \u0026amp; F \u0026amp; F \u0026amp; T \u0026amp; T \u0026amp; F \u0026amp; F \\\\ F \u0026amp; T \u0026amp; F \u0026amp; T \u0026amp; T \u0026amp; T \u0026amp; F \\\\ F \u0026amp; F \u0026amp; F \u0026amp; F \u0026amp; F \u0026amp; T \u0026amp; T\\end{array}$$\u003c/p\u003e\n\u003cp\u003eThe only way to make the conditional statement $p → q$ false is to have $p$ be true and $q$ be false.\u003c/p\u003e\n\u003ch3 id=\"important-logical-equivalences\"\u003e\n \u003cspan\u003eImportant logical equivalences\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#important-logical-equivalences\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003e$$\\begin{aligned}p \\leftrightarrow q \\quad \u0026amp;\\not\\equiv \\quad p \\wedge q \\\\ \\neg(p \\leftrightarrow q) \\quad \u0026amp;\\equiv \\quad p \\oplus q \\\\ p \\rightarrow q \\quad \u0026amp;\\equiv \\quad \\neg p \\vee q \\\\ p \\leftrightarrow q \\quad \u0026amp;\\equiv \\quad q \\leftrightarrow p \\\\ p \\rightarrow q \\quad \u0026amp;\\not\\equiv \\quad q \\rightarrow p \\\\ p \\rightarrow q \\quad \u0026amp;\\not\\equiv \\quad \\neg p \\rightarrow \\neg q \\\\ p \\rightarrow q \u0026amp;\\equiv \\quad \\neg q \\rightarrow \\neg p\\end{aligned}$$\u003c/p\u003e\n\u003ch2 id=\"predicates\"\u003e\n \u003cspan\u003ePredicates\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#predicates\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eA \u003cstrong\u003epredicate\u003c/strong\u003e is a function from a given set (domain) to ${T , F }$.\u003c/p\u003e\n\u003cp\u003eA predicate can be applied, or \u003cstrong\u003eevaluated\u003c/strong\u003e at, an element of the domain.\u003c/p\u003e\n\u003cp\u003eTwo predicates over the same domain are \u003cstrong\u003eequivalent\u003c/strong\u003e means they evaluate to the same truth values for all possible assignments of domain elements to the input.\u003c/p\u003e\n\u003cp\u003eThe \u003cstrong\u003etruth set\u003c/strong\u003e of a predicate is the collection of all elements in its domain where the predicate evaluates to $T$.\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eNotation:\u003c/em\u003e for a predicate $P$ with domain $X_{1} \\times \\cdots \\times X_{n}$ and a $n$-tuple $((x_{1}, \\ldots, x_{n}))$ with each $x_{i} \\in X$, we write $P((x_{1}, \\ldots, x_{n}))$ to mean $P((((x_{1}, \\ldots, x_{n}))))$.\u003c/p\u003e\n\u003ch2 id=\"quantifiers\"\u003e\n \u003cspan\u003eQuantifiers\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#quantifiers\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eThe \u003cstrong\u003euniversal quantification\u003c/strong\u003e of $P (x)$ is the statement “$P (x)$ for all values of $x$ in the domain” and is written $∀xP (x)$. An element for which $P (x) = F$ is called a counterexample of $∀xP (x)$.\u003c/p\u003e\n\u003cp\u003eThe \u003cstrong\u003eexistential quantification\u003c/strong\u003e of $P (x)$ is the statement “There exists an element $x$ in the domain such that $P (x)$” and is written $∃xP (x)$. An element for which $P (x) = T$ is called a witness of $∃xP (x)$.\u003c/p\u003e\n\u003ch3 id=\"quantifier-version-of-de-morgans-law\"\u003e\n \u003cspan\u003eQuantifier version of De Morgan\u0026rsquo;s law\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#quantifier-version-of-de-morgans-law\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003e$\\neg \\forall x P(x) \\equiv \\exists x(\\neg P(x))$\u003c/li\u003e\n\u003cli\u003e$\\neg \\exists x Q(x) \\equiv \\forall x(\\neg Q(x))$\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch1 id=\"proof\"\u003e\n \u003cspan\u003eProof\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#proof\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h1\u003e\n\u003ch2 id=\"strategies\"\u003e\n \u003cspan\u003eStrategies\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#strategies\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003ch3 id=\"basics\"\u003e\n \u003cspan\u003eBasics\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#basics\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003eTo prove that $∀xP (x)$ is true: evaluate $P(x)$ at each domain element to confirm it is $T$.\u003c/li\u003e\n\u003cli\u003eTo prove that $∀xP (x)$ is false: we find a counterexample: an element in the domain for which $P(x)$ is false.\u003c/li\u003e\n\u003cli\u003eTo prove that $∃xP (x)$ is true: we find a witness: an element in the domain for which $P(x)$ is true.\u003c/li\u003e\n\u003cli\u003eTo prove that $∃xP (x)$ is false: evaluate $P(x)$ at each domain element to confirm it is $F$. In other words, it is equivalent to prove $∀x ¬P (x)$.\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch3 id=\"proof-of-universal-by-exhaustion\"\u003e\n \u003cspan\u003eProof of universal by exhaustion\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#proof-of-universal-by-exhaustion\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eTo prove that $∀x P (x)$ is true when $P$ has a finite domain, evaluate the predicate at each domain element to confirm that it is always $T$.\u003c/p\u003e\n\u003ch3 id=\"proof-by-universal-generalization\"\u003e\n \u003cspan\u003eProof by universal generalization\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#proof-by-universal-generalization\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eTo prove that $∀x P (x)$ is true, we can take an arbitrary element $e$ from the domain and show that $P (e)$ is true, without making any assumptions about $e$ other than that it comes from the domain.\u003c/p\u003e\n\u003ch3 id=\"evidence-for-conjunction\"\u003e\n \u003cspan\u003eEvidence for conjunction\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#evidence-for-conjunction\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eTo prove that $p ∧ q$ is true, have two subgoals: subgoal (1) prove $p$ is true; and, subgoal (2) prove $q$ is true.\nTo prove that $p ∧ q$ is false, it’s enough to prove that $p$ is false.\u003c/p\u003e\n\u003cp\u003eTo prove that $p ∧ q$ is false, it’s enough to prove that $q$ is false.\u003c/p\u003e\n\u003ch3 id=\"proof-by-cases\"\u003e\n \u003cspan\u003eProof by cases\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#proof-by-cases\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eTo prove $q$, if we know that $p_1 ∨ p_2$ is true, and we can show that $(p_1 → q)$ is true and we can show that $(p_2 → q)$, then we can conclude $q$ is true\u003c/p\u003e\n\u003ch3 id=\"proof-of-conditional-by-direct-proof\"\u003e\n \u003cspan\u003eProof of conditional by direct proof\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#proof-of-conditional-by-direct-proof\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eTo prove that the conditional statement $p → q$ is true, we can assume $p$ is true and use that assumption to show $q$ is true.\u003c/p\u003e\n\u003ch3 id=\"proof-of-conditional-by-contrapositive-proof\"\u003e\n \u003cspan\u003eProof of conditional by contrapositive proof\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#proof-of-conditional-by-contrapositive-proof\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eTo prove that the implication $p → q$ is true, we can assume $q$ is false and use that assumption to show $p$ is also false.\u003c/p\u003e\n\u003ch3 id=\"proof-by-contradiction\"\u003e\n \u003cspan\u003eProof by Contradiction\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#proof-by-contradiction\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eTo prove that a statement $p$ is true, pick another statement $r$ and once we show that $¬p → (r ∧ ¬r)$ then we can conclude that $p$ is true.\u003c/p\u003e\n\u003ch2 id=\"induction\"\u003e\n \u003cspan\u003eInduction\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#induction\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003ch3 id=\"proof-by-structural-induction\"\u003e\n \u003cspan\u003eProof by structural Induction\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#proof-by-structural-induction\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eTo prove a universal quantification over a recursively defined set:\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eBasis Step:\u003c/strong\u003e Show the statement holds for elements specified in the basis step of the definition.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eRecursive Step:\u003c/strong\u003e Show that if the statement is true for each of the elements used to construct new elements in the recursive step of the definition, the result holds for these new elements.\u003c/p\u003e\n\u003ch3 id=\"proof-by-mathematical-induction\"\u003e\n \u003cspan\u003eProof by Mathematical Induction\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#proof-by-mathematical-induction\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eTo prove a universal quantification over the set of all integers greater than or equals some base integer $b$:\n\u003cstrong\u003eBasis Step:\u003c/strong\u003e Show the statement holds for b.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eRecursive Step:\u003c/strong\u003e Consider an arbitrary integer n greater than or equal to $b$, assume (as the \u003cstrong\u003einduction hypothesis\u003c/strong\u003e) that the property holds for $n$, and use this and other facts to prove that the property holds for $n + 1$.\u003c/p\u003e\n\u003ch3 id=\"proof-by-strong-induction\"\u003e\n \u003cspan\u003eProof by Strong Induction\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#proof-by-strong-induction\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eTo prove that a universal quantification over the set of all integers greater than or equal to some base integer $b$ holds, pick a fixed nonnegative integer $j$ and then:\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eBasis Step\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eShow the statement holds for $b, b + 1, . . . , b + j$.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eRecursive Step\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eConsider an arbitrary integer $n$ greater than or equal to $b + j$, assume (as the strong\ninduction hypothesis) that the property holds for each of $b, b + 1, . . . , n$, and use this and other facts to prove that the property holds for $n + 1$.\u003c/p\u003e\n\u003ch1 id=\"cardinality\"\u003e\n \u003cspan\u003eCardinality\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#cardinality\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h1\u003e\n\u003ch2 id=\"functions-and-comparing-cardinalities\"\u003e\n \u003cspan\u003eFunctions and comparing cardinalities\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#functions-and-comparing-cardinalities\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eLet $D$ and $C$ be nonempty sets. A \u003cstrong\u003efunction\u003c/strong\u003e $f$ from $D$ (\u003cstrong\u003edomain\u003c/strong\u003e) to $C$ (\u003cstrong\u003ecodomain\u003c/strong\u003e) is an assignment of one element of $C$ to each element of $D$.\u003c/p\u003e\n\u003ch3 id=\"one-to-one-function\"\u003e\n \u003cspan\u003eOne-to-one Function\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#one-to-one-function\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eA function $f : D → C$ is \u003cstrong\u003eone-to-one\u003c/strong\u003e (or \u003cstrong\u003einjective\u003c/strong\u003e) means for every $a$, $b$ in the domain $D$, if $f (a) = f (b)$ then $a = b$.\u003c/p\u003e\n\u003cp\u003eFor sets $A$, $B$, we say that the \u003cstrong\u003ecardinality of $A$ is no bigger than the cardinality of $B$\u003c/strong\u003e, and write $|A| ≤ |B|$, to mean there is a one-to-one function with domain $A$ and codomain $B$.\u003c/p\u003e\n\u003cp\u003e$|A| \\leq|B|$ means $\\exists f: A \\rightarrow B \\forall a_{1} \\in A \\forall a_{2} \\in A((a_{1} \\neq a_{2} \\rightarrow f((a_{1})) \\neq f((a_{2}))))$\u003c/p\u003e\n\u003ch3 id=\"onto-function\"\u003e\n \u003cspan\u003eOnto function\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#onto-function\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eA function $f : D → C$ is \u003cstrong\u003eonto\u003c/strong\u003e (or \u003cstrong\u003esurjective\u003c/strong\u003e) means for every $b$ in the codomain, there is an element $a$ in the domain with $f (a) = b$.\u003c/p\u003e\n\u003cp\u003eFormally, $f : D → C$ is onto means $∀b ∈ C \\exists a ∈ D (f (a) = b)$.\u003c/p\u003e\n\u003cp\u003eFor sets $A$, $B$, we say that the \u003cstrong\u003ecardinality of $A$ is no smaller than the cardinality of $B$\u003c/strong\u003e, and write $|A| ≥ |B|$, to mean there is an onto function with domain $A$ and codomain $B$.\u003c/p\u003e\n\u003cp\u003e$|A|\\ge|B|$ means $\\exists f: A \\rightarrow B \\forall b \\in B \\exists a \\in A(f(a)=b)$\u003c/p\u003e\n\u003ch3 id=\"bijection\"\u003e\n \u003cspan\u003eBijection\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#bijection\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eA function $f : D → C$ is a bijection means that it is both one-to-one and onto. The inverse of a bijection $f : D → C$ is the function $g : C → D$ such that $g(b) = a \\text{ iff } f (a) = b$.\u003c/p\u003e\n\u003cp\u003e$|A|=|B|$ means$\\exists f: A \\rightarrow B \\forall b \\in B \\exists a \\in A((f(a)=b \\wedge \\forall a^{\\prime} \\in A((a \\neq a^{\\prime} \\rightarrow f((a^{\\prime})) \\neq b))))$\u003c/p\u003e\n\u003ch3 id=\"cantor-schroder-bernstein-theorem\"\u003e\n \u003cspan\u003eCantor-Schroder-Bernstein Theorem\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#cantor-schroder-bernstein-theorem\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003e$|A|=|B| \\text { iff } (|A| \\leq|B| \\text { and }|B| \\leq|A|) \\text { iff } (|A| \\geq|B| \\text { and }|B| \\geq|A|)$\u003c/p\u003e\n\u003cp\u003eTo prove $|A| = |B|$, we can do any one of the following\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eProve there exists a bijection $f : A → B$;\u003c/li\u003e\n\u003cli\u003eProve there exists a bijection $f : B → A$;\u003c/li\u003e\n\u003cli\u003eProve there exists two functions $f_1 : A → B$, $f_2 : B → A$ where each of $f_1$, $f_2$ is one-to-one.\u003c/li\u003e\n\u003cli\u003eProve there exists two functions $f_1 : A → B$, $f_2 : B → A$ where each of $f_1$, $f_2$ is onto.\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"properties-of-cardinality\"\u003e\n \u003cspan\u003eProperties of cardinality\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#properties-of-cardinality\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003e$$\\begin{array}{l}\\forall A(|A|=|A|) \\\\ \\forall A \\forall B(|A|=|B| \\rightarrow|B|=|A|) \\\\ \\forall A \\forall B \\forall C((|A|=|B| \\wedge|B|=|C|) \\rightarrow|A|=|C|)\\end{array}$$\u003c/p\u003e\n\u003ch2 id=\"countably-infinite-sets\"\u003e\n \u003cspan\u003eCountably infinite sets\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#countably-infinite-sets\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eA set $A$ is \u003cstrong\u003ecountably infinite\u003c/strong\u003e means it is the same size as $\\mathbb N$.\u003c/p\u003e\n\u003cp\u003eAn infinite set is countable if and only if it is possible to list the elements of the set in a sequence\u003c/p\u003e\n\u003cp\u003eExamples of countably infinite sets: $S, L, \\Z, \\Z^+, \\Z , \\Z × \\Z$.\u003c/p\u003e\n\u003ch2 id=\"uncountable-sets\"\u003e\n \u003cspan\u003eUncountable sets\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#uncountable-sets\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eAn \u003cstrong\u003euncountable\u003c/strong\u003e set is a set that is not finite and is not countably infinite.\u003c/p\u003e\n\u003ch3 id=\"cantors-diagonal-argument\"\u003e\n \u003cspan\u003eCantor\u0026rsquo;s diagonal argument\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#cantors-diagonal-argument\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003e\u003cstrong\u003eClaim\u003c/strong\u003e: $|\\mathbb{N}|\u0026lt;|\\mathcal{P}(\\mathbb{N})|$\n\u003cstrong\u003eProof\u003c/strong\u003e: Towards a proof by universal generalization, consider an arbitrary function $f: \\mathbb{N} \\rightarrow \\mathcal{P}(\\mathbb{N})$. We want to prove that $f$ is not onto.\nRewriting using the definition of onto:\u003c/p\u003e\n\u003cp\u003e$$\\neg(\\forall B \\in \\mathcal{P}(\\mathbb{N}) \\exists a \\in \\mathbb{N}(f(a)=B))$$\u003c/p\u003e\n\u003cp\u003eBy logical equivalence, we can write this as an existential statement:\u003c/p\u003e\n\u003cp\u003e$$\\exists B \\in \\mathcal{P}(\\mathbb{N}) \\forall a \\in \\mathbb{N}(f(a) \\neq B)$$\u003c/p\u003e\n\u003cp\u003eIn search of a witness, define the following collection of nonnegative integers:\u003c/p\u003e\n\u003cp\u003e$$D_{f}={n \\in \\mathbb{N} \\mid n \\notin f(n)}$$\u003c/p\u003e\n\u003cp\u003eBy definition of power set, since all elements of $D_{f}$ are in $\\mathbb{N}$, it follows that\u003c/p\u003e\n\u003cp\u003e$$D_{f} \\in \\mathcal{P}(\\mathbb{N})$$\u003c/p\u003e\n\u003cp\u003eTherefore, it\u0026rsquo;s enough to prove the following lemma:\n\u003cstrong\u003eLemma\u003c/strong\u003e: $\\forall a \\in \\mathbb{N}((f(a) \\neq D_{f}))$\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eProof of lemma:\u003c/strong\u003e Towards universal generalization, consider an arbitrary $a \\in \\mathbb{N}$. By definition of set equality, we want to prove that $\\exists x((\\neg((x \\in f(a) \\leftrightarrow x \\in D_{f}))))$. For a witness, consider $x=a$. There are two cases: $a \\in f(a) \\vee a \\notin f(a)$. By definition of $D_{f}$, each guarantees that $f(a) \\neq D_{f}$.\u003c/p\u003e\n\u003cp\u003eBy the Lemma, we have proved that $f$ is not onto, and since $f$ was arbitrary, there are no onto functions from $\\mathbb{N}$ to $\\mathcal{P}(\\mathbb{N})$.\u003c/p\u003e\n\u003ch3 id=\"the-set-of-real-numbers\"\u003e\n \u003cspan\u003eThe set of real numbers\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#the-set-of-real-numbers\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003e$\\mathbb R$ is \u003cstrong\u003euncountable\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eReflexiviy\u003c/strong\u003e ⇒ $\\forall a \\in \\mathbb{R}(a \\leq a)$\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAntisymmetry\u003c/strong\u003e ⇒ $\\forall a \\in \\mathbb{R} \\forall b \\in \\mathbb{R}((a \\leq b \\wedge b \\leq a) \\rightarrow(a=b))$\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTransitivity\u003c/strong\u003e ⇒ $\\forall a \\in \\mathbb{R} \\forall b \\in \\mathbb{R} \\forall c \\in \\mathbb{R}((a \\leq b \\wedge b \\leq c) \\rightarrow(a \\leq c))$\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTrichotomy\u003c/strong\u003e ⇒ $\\forall a \\in \\mathbb{R} \\forall b \\in \\mathbb{R}((a=b \\vee b\u0026gt;a \\vee a\u0026lt;b)$\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eLeast upper bound\u003c/strong\u003e ⇒ Every nonempty set of real numbers that is bounded above has a least upper bound\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eNested intervals\u003c/strong\u003e ⇒ For each sequence of intervals $[a_n, b_n]$ where, for each $n$, $a_n \u0026lt; a_n+1 \u0026lt; b_n+1 \u0026lt; b_n$, there is at least one real number $x$ such that, for all $n$, $a_n ≤ x ≤ b_n$.\u003c/p\u003e\n\u003ch1 id=\"relation\"\u003e\n \u003cspan\u003eRelation\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#relation\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h1\u003e\n\u003cp\u003eWhen $A$ and $B$ are sets, we say any subset of $A × B$ is a \u003cstrong\u003ebinary relation\u003c/strong\u003e. There are other ways to represent a relation $R$.\u003c/p\u003e\n\u003cp\u003eWhen $A$ is a set, we say any subset of $A × A$ is a (binary) \u003cstrong\u003erelation\u003c/strong\u003e on $A$.\u003c/p\u003e\n\u003cp\u003eLet $R_{(\\bmod\\ n)}$ be the set of all pairs of integers $(a, b)$ such that $(a \\bmod n = b \\bmod n)$. Then $a$ is congruent to $b$ mod $n$ means $(a, b) ∈ R_{(\\bmod\\ n)}$. A common notation is to write this as $a ≡ b(\\bmod\\ n)$.\u003c/p\u003e\n\u003cp\u003eA relation $R$ on a set $A$ is called \u003cstrong\u003ereflexive\u003c/strong\u003e means $(a, a) ∈ R$ for every element $a ∈ A$. A relation $R$ on a set $A$ is called \u003cstrong\u003esymmetric\u003c/strong\u003e means $(b, a) ∈ R$ whenever $(a, b) ∈ R$, for all $a, b ∈ A$. A relation $R$ on a set $A$ is called \u003cstrong\u003etransitive\u003c/strong\u003e means whenever $(a, b) ∈ R$ and $(b, c) ∈ R$, then $(a, c) ∈ R$, for all $a, b, c ∈ A$.\u003c/p\u003e\n\u003ch2 id=\"equivalence-relations\"\u003e\n \u003cspan\u003eEquivalence relations\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#equivalence-relations\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eA relation is an equivalence relation means it is \u003cstrong\u003ereflexive\u003c/strong\u003e, \u003cstrong\u003esymmetric\u003c/strong\u003e, and \u003cstrong\u003etransitive\u003c/strong\u003e.\u003c/p\u003e\n\u003cp\u003eAn equivalence class of an element $a ∈ A$ for an equivalence relation $R$ on the set $A$ is the set ${s ∈ A|(a, s) ∈ R}$. We write this as $[a]_R$.\u003c/p\u003e\n\u003cp\u003eA partition of a set $A$ is a set of non-empty, disjoint subsets $A_1, A_2, · · · , A_n$ such that $A_1 ∪ A_2 ∪ · · · ∪ A_n = A$.\u003c/p\u003e\n\u003cp\u003eWe can partition a set using equivalence classes.\u003c/p\u003e\n\u003ch2 id=\"modular-arithmetic\"\u003e\n \u003cspan\u003eModular arithmetic\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#modular-arithmetic\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eFor $a, b ∈ \\Z$ and positive integer $n, (a, b) ∈ R(\\bmod\\ n)$ if and only if $n|a-b$.\u003c/p\u003e\n\u003cp\u003eFor $a, b ∈ \\Z$ and positive integer $n$, if $a ≡ b(\\bmod\\ n)$ and $c ≡ d(\\bmod\\ n)$ then $a + c ≡ b + d(\\bmod\\ n)$ and $ac ≡ bd(\\bmod\\ n)$. \u003cem\u003eInformally\u003c/em\u003e: can bring mod “inside” and do it first, for addition and for multiplication.\u003c/p\u003e\n\u003cp\u003e$(a ⋅ b) \\bmod m = [(a \\bmod m) ⋅ (b \\bmod m)] \\bmod m$\u003c/p\u003e\n\u003cp\u003e$(a^b\\bmod c)^n\\bmod c=a^{bn}\\bmod c$ (basis of \u003cstrong\u003eDiffie-Hellman key exchange\u003c/strong\u003e)\u003c/p\u003e\n","date_modified":"2026-05-30T00:37:17-07:00","date_published":"2020-12-10T00:00:00Z","id":"https://www.xiyan.dev/writings/2020/12/10/intro-discrete-mathematics-notes/","summary":"Personal notes for CSE 20 Discrete Mathematics for Computer Science, taken Fall 2020 with Shachar Lovett, covering recursive definitions, sets, and proof techniques.","tags":["Math","Computer Science","Course"],"title":"Intro Discrete Mathematics Notes","url":"https://www.xiyan.dev/writings/2020/12/10/intro-discrete-mathematics-notes/"},{"content_html":"\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003eWARNING:\u003c/strong\u003e This article was written by the author during high-school, in a\nnon-professional capacity.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eMeta-learning, or learning to learn, is a paradigm of machine learning\nalgorithms that can generalize itself with meta-knowledge of a certain form such that it can apply to various settings. While it is originally a hallmark of human intelligence, numerous meta-learning perspectives and approaches are springing up in recent years. This paper provides an overview of recent meta-learning approaches, especially for Model-Agnostic Meta-Learning (and its derivatives), Meta-Reinforcement Leaning, and Few-shot (or Zero/One-shot), three emerging methods in the past five years. Model-Agnostic Meta-Learning is a meta-learning algorithm compatible with any model trained using the gradient descent; Meta-Reinforcement refers to doing meta-learning in the field of meta-learning as to develop RL agents that can solve novel tasks fast and efficiently; and Few-shot learning is a few-sample specialization of meta-learning. Their mechanisms, applications, and performances are thoroughly analyzed.\u003c/p\u003e\n\u003cp\u003e\u003cem\u003e\u003cstrong\u003eKeywords\u003c/strong\u003e\u003c/em\u003e: Meta-Learning, MAML, Meta-Reinforcement Learning, Few-Shot\nLearning\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eIf they can, computers would certainly envy that Homo sapiens are all gifted learners. The ability of learning comes to us so naturally that we are apt to forget what a miracle it is. Kids learn by imitating their parents, high schoolers get to know historical events across space and time from their teacher’s preaching, and every human being unconsciously learns from experiences be it sad or happy. In fact, not only do humans acquire concrete knowledge through education or behavioristic processes of trials and errors, but more importantly, we learn how to learn — yet, this is what computers are struggling with so far. \u0026ldquo;Learning to learn\u0026rdquo; was firstly termed \u0026ldquo;meta-learning\u0026rdquo; by Donald B. Maudsley in educational psychology, which John Biggs described as \u0026ldquo;being aware of and taking control of one\u0026rsquo;s own learning\u0026rdquo; (Biggs, 1985). As computers may marvel at this hallmark of human intelligence, the concept meta-learning emerged in the field of machine learning (ML), a discipline concentrated on computer\u0026rsquo;s learning.\u003c/p\u003e\n\u003cp\u003eIn the context of machine learning, meta-learning, according to Brazdil, refers to the \u0026ldquo;study of principled methods that exploit meta-knowledge to obtain efficient models and solutions by adapting machine learning and data mining processes\u0026rdquo; (Brazdil \u003cem\u003eet al.\u003c/em\u003e, 2009). Though multiple other definitions exist, the central, underlying idea is that the meta learning algorithm should be able to \u003cem\u003egeneralize\u003c/em\u003e itself with experience such that it can apply to various scenarios. It may challenge people\u0026rsquo;s common stereotype that an algorithm is oriented towards a fixed, procedural problem, or at least it should subject to the training data if it comes after \u0026ldquo;machine learning\u0026rdquo;; however, in the last 20 years, meta-learning did succeed in that.\u003c/p\u003e\n\u003cp\u003eIn the past decade, the focus of machine learning research had shifted from deriving specific solutions for a single setting to general solutions for various settings, and future approaches of machine learning would implement even greater algorithmic generality. Deep convolutional neural networks like the famous AlexNet could well handle the training rate and overfitting problems while reaching a considerable success rate (Krizhevsky \u003cem\u003eet al.\u003c/em\u003e, 2017); however, that could only be trained toward a single task. To tackle this limitation, studies have proposed reinforcement learning to leverage more available training data. This approach proved itself sound when incorporated into AlphaGo, whose ability to learn in a behavioral manner stunned the public (Silver \u003cem\u003eet al.\u003c/em\u003e, 2016). Nonetheless, while deep reinforcement learning is succeeds in some structures like in the chessboard, its ability to generalize still hinges on predefined task guidelines.\u003c/p\u003e\n\u003cp\u003eFacing the “arbitrary, intrinsically-complex, outside world” that ML algorithms need to tackle with, Rich Sutton observed that, instead of trying to build simple ways to model the content of minds, “we should build in only the meta-methods that can find and capture this arbitrary complexity” (2019). As it turns out, the concept of meta-learning seems a panacea. Although its definition not yet unified, meta-learning is widely recognized as one of the most promising machine learning paradigms in future artificial intelligence advancement. For example, it has been witnessed that a robot hand could solve the Rubik\u0026rsquo;s cube without a hardcoded algorithm; what supports this astonishing feat was a technique called Automatic Domain Randomization (ADR) — and that was dictated by a central meta-learning algorithm (OpenAI \u003cem\u003eet al.\u003c/em\u003e, 2019). Innovations on meta-learning are still undergoing an explosion. To name a few, there are metric-based, model-based, and optimization-based approaches, as well as combinations of other existing methods like meta-reinforcement learning. Each of them carries a special perspective to this curious branch of machine learning, and each of them makes the next revolution of artificial intelligence closer to sight.\u003c/p\u003e\n\u003cp\u003eGiven the increasing complexity in meta-learning research, this paper is intended to serve as a survey of recent meta-learning approaches. After a brief explanation of machine learning concepts and terminologies, literature of three state-of-the-art meta-learning areas, Model-Agnostic Meta-Learning (MAML), Meta-Reinforcement Learning (MRL), and Few-shot (or Zero/One-shot) learning and their respective perspectives and applications will be particularly analyzed. Considering meta-learning\u0026rsquo;s missions, the analysis focuses on three specific dimensions: training difficulty, generalizability, and accuracy.\u003c/p\u003e\n\u003ch2 id=\"preliminaries\"\u003e\n \u003cspan\u003ePreliminaries\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#preliminaries\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eAlthough meta-learning is a broad concept that spans over a variety of machine learning methods and tasks, understanding the principles of those \u003cem\u003ebase-learners\u003c/em\u003e (i.e. the lower-level machine learning algorithms where meta-learning is applied on, usually with a fixed a priori) remains important. A brief overview of some machine learning terminologies and common tasks with respective methods is as follows.\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eMachine learning\u003c/em\u003e, the general term, refers to a set of methods that can detect patterns in data automatically, and then use the uncovered patterns to predict future data or to perform other kinds of decision making under uncertainty (Murphy, 2012, p. 4). To train a machine learning model such that it works as intended, a \u003cem\u003edataset\u003c/em\u003e in a certain form is always necessary. Every record in the dataset is an \u003cem\u003einstance\u003c/em\u003e or a \u003cem\u003esample\u003c/em\u003e, and its properties are known as \u003cem\u003eattributes, features\u003c/em\u003e or \u003cem\u003ecovariates\u003c/em\u003e, whose value called attribute values.\u003c/p\u003e\n\u003cp\u003eIn a \u003cem\u003epredictive\u003c/em\u003e or \u003cem\u003esupervised\u003c/em\u003e learning approach, the \u003cem\u003elearning algorithm\u003c/em\u003e takes in a labeled dataset of input-output pairs, a training \u003cem\u003eset\u003c/em\u003e, and gives out \u003cem\u003eresponse variables\u003c/em\u003e of similar forms. When the response variable is categorical, the problem is known as \u003cem\u003eclassification\u003c/em\u003e or \u003cem\u003epattern recognition.\u003c/em\u003e Common supervised classification methods are linear classifiers, support vector machines (Boser \u003cem\u003eet al.\u003c/em\u003e, 1992), decision trees (Quinlan, 1986), naive Bayes classifiers (Jensen, 1996), and neural networks. On the other hand, if the response variable is quantitative, the problem is known as regression. Neural networks, random forest (Breiman, 2001), as well as polynomial regression, can apply to it.\u003c/p\u003e\n\u003cp\u003eWhereas in a \u003cem\u003edescriptive\u003c/em\u003e or \u003cem\u003eunsupervised learning\u003c/em\u003e setting, the dataset is not labeled (i.e. unpaired). It runs automatically to discover interesting patterns. Its common applications are cluster analysis, association rule, and dimensionality reduction, where methods like K-Nearest Neighbors (KNN), K-means, and hierarchical clustering can apply (Mucherino \u003cem\u003eet al.\u003c/em\u003e, 2009). They have better performances in terms of generalizability but usually lack accuracy.\u003c/p\u003e\n\u003cp\u003eAnother type of learning is Reinforcement Learning (RL). It differs from supervised learning in not needing a complete paired dataset, and it learns much like a kid, through behavior, trial-and-error processes. Reinforcement learning is closer to how a human learns; it is also closely related to meta-learning. Examples of it are Q-learning and SARSA, both of which incorporate a state-action-reward-state process.\u003c/p\u003e\n\u003cp\u003eEvaluating a trained model requires a validation set, and processes like holding-out, cross-validation, and bootstrapping are usually adopted. The tests on validation sets give metrics like classification accuracy, confusion matrix (more detailed), logarithmic loss, Area under Curve (AUC), and F-measures for classification models, as well as Mean Absolute Error (MEA) and Root Mean Squared Error (RMSE) for regression models. Underfitting and overfitting are two major obstacles for successful learning, the former unable to sufficiently detect patterns in the training set while the latter learning too well on the training set but failing at the validating set.\u003c/p\u003e\n\u003ch2 id=\"a-literature-survey\"\u003e\n \u003cspan\u003eA Literature Survey\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#a-literature-survey\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eThis section provides a review of recent meta-learning approaches, especially for Model-Agnostic Meta-Learning (and its derivatives), Meta-Reinforcement Leaning, and Few-shot (or Zero/One-shot), three rising methods in the past five years. The reviews also cover the perspectives they rely on as well as possible applications they have enabled.\u003c/p\u003e\n\u003cp\u003eGranted, there are plentiful historical meta-learning approaches whose performance excel in certain areas – for example, bagging (Breiman, 1996) and boosting (Freund \u0026amp; Schapire, 1997) attempted at exploiting variations in data and are considered meta-learning methods by Brazdil \u003cem\u003eet al.\u003c/em\u003e (2009) – and many other algorithms implement the idea of meta-learning while not taking this name. In fact, the arguably first meta-learning system was studied by Rice (1976) as “the algorithm selection problem”, which has evolved into the meta-learning area of “algorithm recommendation”. Besides, a few more explored areas are dynamic bias selection, inductive transfer, and holistic meta-learning system (Lemke \u003cem\u003eet al.\u003c/em\u003e, 2015).\u003c/p\u003e\n\u003cp\u003eHowever, most of the earlier methods of these areas are fully studied in previous literature reviews, as by Bhatt \u003cem\u003eet al.\u003c/em\u003e (2012), Brazdil \u003cem\u003eet al.\u003c/em\u003e (2009), Lemke \u003cem\u003eet al.\u003c/em\u003e (2015), Vanschoren, (2018), and Vilalta \u0026amp; Drissi (2002). Hence, this review instead concentrates on providing an updated survey on the selected new state-of-the-arts – starting with MAML.\u003c/p\u003e\n\u003ch3 id=\"model-agnostic-meta-learning-maml\"\u003e\n \u003cspan\u003eModel-Agnostic Meta-Learning (MAML)\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#model-agnostic-meta-learning-maml\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eFirst proposed by Finn \u003cem\u003eet al.\u003c/em\u003e (2017), the Model-Agnostic Meta-Learning approach was an algorithm compatible with any model trained using the gradient descent method and applicable to problems including but not limited to classification, regression, and reinforcement learning. This algorithm aims to achieve rapid adaptation and can train a model’s parameters with few gradient updates as to shorten the learning time on a new task. Based on the intuition that some internal representations of the data are more transferrable than others, the method defies the conventional wisdom – not ingesting entire datasets and feature embeddings.\u003c/p\u003e\n\u003cp\u003eAs for the specific implementation, it considers a model represented by a parametrized function, whose model parameters alter when encountering new tasks. Then, the meta-optimization performs on these parameters using the update calculated via stochastic gradient descent (SGD). It is noted that this MAML meta-gradient update involves a gradient through a gradient.\u003c/p\u003e\n\u003cp\u003eAs is formally proved by the authors, the algorithm applies to supervised regression and classification, as well as to reinforcement learning (where it enables quick acquisition of policies for new tasks through limited experience). For its application in regression, experimental evaluations have shown that MAML optimizes the model parameters into a region amenable to rapid adaptions and sensitive to loss functions, without overfitting to the parameters; this enables improvements over one step. Also, for classification tasks, evaluations around image recognition on the Omniglot and MiniImagenet datasets indicates that the learned convolutional model compares well to the state-of-the-art results.\u003c/p\u003e\n\u003cp\u003eHowever, this method uses fewer overall parameters than the meta-learner Long Short-Term Memory (LSTM), and it outperforms LSTM on 5-way classifications for both datasets. Limitations of it, nevertheless, are that it represents a narrower scope of application than prior memory-augmented neural networks and that it has a critical computational expense at deriving the second derivatives when backpropagating the meta-gradient in the meta-objective. Besides, its application in reinforcement learning is proven to be sound. Results in the 2D navigation and Locomotion challenges show that MAML can learn a model with a quick single gradient update (better initialization) while continuing to improve with future updates.\u003c/p\u003e\n\u003cp\u003eThe fundamental notion of MAML is to adapt to a large number of tasks and thus gain better generalizability. The core optimizing mechanism, meta-gradient update, was also applied by Andrychowicz \u003cem\u003eet al.\u003c/em\u003e (2016), where the algorithm is “learning to learn by gradient descent by gradient”, and similarly, it utilized LSTMs as part of the optimization. Therefore, Andrychowicz’s algorithm is worth understanding as well.\u003c/p\u003e\n\u003cp\u003eTo construct learning algorithms that perform well on a cluster of optimization problems, Andrychowicz \u003cem\u003eet al.\u003c/em\u003e proposed such an algorithm to cast the challenge of transfer learning as one of generalization. By directly parameterizing the optimizer and applying the gradient descent method, the meta-optimizer can minimize the loss of a given function. Applying recurrent neural networks requires the algorithm to optimize a vast number of parameters, thus Andychowicz incorporates a \u003cem\u003ecoordinatewise network architecture\u003c/em\u003e (which involves an optimizer that operates on the parameters of the objective function) to simplify the process to a single coordinate. The optimizer is implemented with a two-layer Long-Short Term Memory (LSTM) network.\u003c/p\u003e\n\u003cp\u003eResulting learning curves show that optimizers learned from this procedure substantially outperforms the baselines across many settings. It exhibits a sloppier learning curve handling quadratic convex optimization and small neural networks on MNIST; can be generalized to different architectures (like MLPs with 40 hidden units, networks with two hidden layers, and networks using ReLU activations); boosts classification performance on the CIFAR-2, -5, and -10 datasets; and enables only 1 style training Neural Art (artistic style transfer using convolutional networks). The evidence all confirms that the learned algorithms, implemented by LSTMs, can outperform their generic, hand-crafted competitors in various tasks.\u003c/p\u003e\n\u003cp\u003eReturning to MAML, despite its monumental achievement, there are a few spaces of improvement on Finn’s original version. According to Antoniou \u003cem\u003eet al.\u003c/em\u003e (2019), though the MAML algorithm had sped up the process of learning and improved generalization performance, it has many issues that render it problematic to put into practice. These include its training instability, second-order derivative cost, absence of batch normalization statistic accumulation, shared (across step) batch normalization bias, shared inner loop learning rate and fixed outer loop learning rate.\u003c/p\u003e\n\u003cp\u003eTo address these problems, Antoniou \u003cem\u003eet al.\u003c/em\u003e (2019) proposed a stable, automated, and improved MAML, which they named MAML++. For each issue mentioned above, they attempted a way to solve it. A Multi-step Loss Optimization (MSL) solves the alleviates the gradient instability by improving the gradient propagation. The Derivative-Order Annealing (DA) technique lowers the second-order derivative cost. Per-Step Batch Normalization Running Statistics (BNRS) solves the absence of batch normalization statistic accumulation and potentially improves its generalization performance. Per-Step Batch Normalization Weights and Biases (BNWB) cancels the shared batch normalization bias whilst increasing the convergence speed, stability, and generalization performance. Finally, Learning Per-Layer Per-Step Learning Rates and Gradient Directions (LSLR) and Cosine Annealing of Meta-Optimizer Learning Rate (CA) solves the previous two learning rate problems, respectively.\u003c/p\u003e\n\u003cp\u003eEmpirical results indicate that this MAML++ marks a new state of the art across all few-shot tasks. Across Omniglot and Mini-Imagenet, MAML++ reduces the inner loop hyperparameter sensitivity, improves the generalization error, and stabilizes and speeds up MAML. While each proposed methodology outperforms MAML individually, BNRS and BNWB contribute to the performance boost the most.\u003c/p\u003e\n\u003cp\u003eBesides the implementation improvements given by MAML++, there’s another adaptive version of MAML from Behl \u003cem\u003eet al.\u003c/em\u003e (2019). To ease the use of MAML with or without significantly parameter tuning to reduce the need for grid search, also to make the algorithm converges in fewer iterations, they proposed an elegant optimization to the original method (named Alpha MAML) by incorporating adaptive tuning of both the learning rate and the meta-learning rate. Based on the hypergradient descent (HD) algorithm, this solution updates learning rates by performing gradient descent alongside the original steps of optimization. It is noted that no extra gradient needs to be computed since the previous gradients can be accessed from extra memory storage.\u003c/p\u003e\n\u003cp\u003eThe performance of Alpha MAML is tested upon few-shot image recognition tasks on the Omniglot dataset. Comparing the behavior of Alpha MAML vs MAML, results show that even the badly picked initial learning rate and meta-learning rate values can be tuned automatically by the online learning rate adaptation scheme of Alpha MAML. It appears that the algorithm does the necessary adjustment to both learning rates in each iteration to optimize the loss. Also, results show that Alpha MAML is less sensitive to the hyperparameter choice one dictates at first, and therefore requires much less fine-tuning.\u003c/p\u003e\n\u003ch3 id=\"meta-reinforcement-learning\"\u003e\n \u003cspan\u003eMeta-Reinforcement Learning\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#meta-reinforcement-learning\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eReinforcement learning algorithms learn through experimental trials and feedbacks, and it enables a behavioralist process that resembles that of human. However, training an RL model is sometimes difficult. Meta-Reinforcement Learning refers to doing reinforcement-learning in the field of meta-learning as to develop RL agents that can solve novel tasks fast and efficiently (Weng, 2019). Though ideas similar this emerged decades ago, for example, in the early meta-learning approach “Learning to Learn Using Gradient Descent” (Hochreiter \u003cem\u003eet al.\u003c/em\u003e, 2001), recent methods embodying this idea are plentiful. Below are some selected approaches that achieved state-of-the-art performances in different areas like robotics, image classification, and physics simulation.\u003c/p\u003e\n\u003cp\u003eDeep networks are generalizable if provided with massive labeled data, but for robots in a controlled laboratory setting, only limited supervision is available. Meanwhile, robots can collect ample data from the unstructured real world. Finn \u003cem\u003eet al.\u003c/em\u003e (2017) formalized this problem as semi-supervised reinforcement learning (SSRL). They consider the problem of semi-supervised reinforcement learning, where a reward function can be evaluated in some small set of labeled MDPs, but the resulting policy must be successful on a larger set of unlabeled MDPs for which the reward function is not known. That it uses experiences from the unlabeled set without access to the reward function makes it distinct.\u003c/p\u003e\n\u003cp\u003eFinn \u003cem\u003eet al.\u003c/em\u003e proposed and evaluated an algorithm for performing SSRL, called semi-supervised skill generalization (S3G). In this method, they trained an RL policy for settings where a reward function is available and then ran an algorithm to simultaneously learn rewards and general policies at wider, unstructured scenarios.\u003c/p\u003e\n\u003cp\u003eConsidering the nature of this algorithm, the experimental evaluations are centered on domains where generalization is vital for success. In tasks like obstacle navigation, 2-link reacher, 2-link reacher with vision, half-cheetah jump, S3G all can improve the generalization of the learned neural network policy. Compared to utilizing supervised regression to reward labels, the inverse-RL objective can achieve higher performance. All these results suggest this method is efficient enough for learning on physical systems such as robots.\u003c/p\u003e\n\u003cp\u003eIn addition to semi-supervised methods, Hsu \u003cem\u003eet al.\u003c/em\u003e (2019) devised an unsupervised meta-learning method that aims to learn a learning procedure applicable to a wide range of novel tasks. Although it somehow digresses from Meta-RL, the underlying principle is enlightening - it allows deriving an unsupervised learning objective that incorporates the usage of representations. Their research shows that learning algorithms acquired through unsupervised meta-learning achieve better downstream performance than the original version without any additional assumptions or supervision. This implies that it is possible to leverage unsupervised embeddings to propose tasks for meta-learning algorithms and create unsupervised methods that are effective for human-specified downstream tasks.\u003c/p\u003e\n\u003cp\u003eIntending to leverage unlabeled data, Hsu \u003cem\u003eet al.\u003c/em\u003e limit accessible data to an untagged set, and approach the problem by framing it as an acquisition process. Specifically, they intended to construct classification tasks from the data and learn how shall the algorithm effectively learn these tasks. The task generation process is critical here; they should be aptly diverse and structured to be fast learned and extended to their human-specified counterparts. Hence, they employ an adjusted \u003cem\u003ek\u003c/em\u003e-means clustering method for grouping data points by salient features and then sample partitions, clusters, embeddings, and permutations of one-shot labels. They call it clustering to automatically construct tasks for unsupervised meta-learning (CACTUs).\u003c/p\u003e\n\u003cp\u003eHsu \u003cem\u003eet al.\u003c/em\u003e empirically demonstrated that such a method improves upon the utility of unsupervised representations in learning downstream, human-specified tasks, holding across benchmark datasets and tasks. However, one particular concern is that the experimental evaluation of MNIST, Omniglot, and miniImageNet exhibits particular structures, perhaps because they are designed for supervised benchmarks.\u003c/p\u003e\n\u003cp\u003eBesides extending the semi-supervised nature of meta-RL, there are other optimizations of Meta-RL. For example, Lan \u003cem\u003eet al.\u003c/em\u003e (2019) improved the conventional meta-RL methodology with task-embedding and shared policy.\u003c/p\u003e\n\u003cp\u003eMeta-learning in reinforcement learning serves as a guide to take actions such that the cumulative reward is maximized in an environment. However, an obstacle in most meta-RL methods is that they fail to adequately and explicitly model the individuality and commonness of tasks. To further investigate the application of meta-learning in RL domains, Lan \u003cem\u003eet al.\u003c/em\u003e proposed to capture shared information across different tasks and specific information on a single task simultaneously. They introduced a new component into the existing meta-RL methodology, named task encoder, and developed a new approach with better performance on training and acting upon novel tasks. Involving a Task Encoder adaption and Shared Policy, this method is named TESP. TESP explicitly models the individuality and commonness of tasks, and, to be specific, it learns a shared policy that characterizes the task commonness to enable a meta-learner to quickly abstract the individuality of varying tasks.\u003c/p\u003e\n\u003cp\u003eLan \u003cem\u003eet al.\u003c/em\u003e evaluated the proposed method on four tasks with the MuJoCo simulator. As it revealed, TESP significantly outperforms all baselines on all four tasks, which indicates a better learning capacity. Moreover, TESP, unlike the baseline models, keeps a good performance on out-of-distribution tasks and avoids overfitting the training distributions. These results combined illustrate that TESP learns not only faster, but also in a way more generalizable.\u003c/p\u003e\n\u003cp\u003eOther optimizations of meta-learning over RL, for example, guiding the policy search, are proposed. Though promising at leveraging past experiences to solve new tasks, Meta-RL algorithms have another general defect that they require a large amount of on-policy experiences during the meta-training process, hence they are subjected to much simpler domains. Noting that meta-reinforcement learning does not entail reinforcement learning during the meta-training process, Mendonca \u003cem\u003eet al.\u003c/em\u003e (2019) proposed to learn a reinforcement learning procedure through imitation of expert policies that solve previously seen tasks. This embodies a meta-RL method that learns fast reinforcement learning via supervised imitation. By employing demonstrations during meta-training, this method can meta-learn adaptation skills with 10x fewer episodes of interactions, and that enables more viable real-world applications.\u003c/p\u003e\n\u003cp\u003eTo reduce total training experience, Mendonca \u003cem\u003eet al.\u003c/em\u003e separated meta-training into two phases — a phase that solves the meta-training task individually and a second phase that utilizes them for meta-learning. In the first phase, the algorithm learns policies for each of the meta-learning tasks, whereas, in the second phase, it learns reinforcingly using these policies as supervision. They named this process Guide Meta-Policy Learn (GMPS).\u003c/p\u003e\n\u003cp\u003eNot only theoretically proven capable of near-optimal cumulative reward when supplied with near-optimal experts, but it has also been experimentally tested that such method allows exploitation of extra supervision more easily. Handling sparse reward tasks, GMPS can adapt to validation tasks better than using a policy pre-trained with MultiTask imitation. For vision-based tasks, GMPS appears more stable and achieves higher rewards than using MAML, just like in the sparse reward case. These results imply its optimistic application in robotics.\u003c/p\u003e\n\u003cp\u003eApart from mere optimizations, Du \u003cem\u003eet al. (2019)\u003c/em\u003e proposed an interesting application of model-based reinforcement learning by learning task-agnostic dynamics priors as to learn task-agnostic dynamics priors from video inputs. It consists of two main steps. A frame prediction model (in this method, SpatialNet) is firstly pretrained. This model is then used to initialize a dynamic model for and RL agent. In the first step, it is required to predict the physical behavior that the model can perform both isolations of entities and accurate modeling of localized spaces as well as interactions. Du \u003cem\u003eet al.\u003c/em\u003e chose a conceptually simple SpatialNet Architecture that is proven not biased and does not account for ego-dynamics. As for the incorporation of a dynamics model into a reinforcement learning setup, the authors combined Mnih’s previous model and the Proximal Policy Optimization (PPO). They call it an Intuitive Physics Agent (IPA).\u003c/p\u003e\n\u003cp\u003eExperiments on physics video datasets suggest that Spatialnet encourages better dynamic generalization and can maintain background details, unlike RCNet. Also, trials on PhysWorld (a collection of 2D physics games) indicate IPA+SpatialNet performs better than JISP, PPO, IPA with RCNet or ConvLSTM. Crucially, IPA encourages the policy to take into account the future physics of entities.\u003c/p\u003e\n\u003ch3 id=\"few-shot-learning\"\u003e\n \u003cspan\u003eFew-Shot Learning\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#few-shot-learning\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h3\u003e\n\u003cp\u003eIn real-world applications, a huge training dataset is not always feasible; that is, machine learning algorithms at times need to be trained upon limited input. For instance, sometimes we need to classify images with only two or three examples per class. However, whereas human can handle these “few-shot” (i.e., learning from a few labeled examples) tasks well through cognitive concepts and prototypes, machines learning methods are still far from human performance.\u003c/p\u003e\n\u003cp\u003eTo accomplish few-shot learning, recent approaches generally utilize additional information from a large base-dataset (Bennequin, 2019). Hence, it is presumable that many few-shot learning methods are indeed meta-learning methods that specialize in few-shot training. As cited by Bennequin, memory-augmented networks, metric learning, gradient-based meta-learners (like MAML), and data generation are examples of popular solutions.\u003c/p\u003e\n\u003cp\u003eAlthough Finn only tested it on supervised regression and classification, and on reinforcement learning tasks, MAML can be extended to many other scenarios necessitating fast adaptation of a Deep Neural Network, like few-shot image recognition. Theoretically applicable, nevertheless, experiments of applying the Model-Agnostic Meta-Learning algorithm to the YOLO detector failed, as doomed by the low prediction of the objectness confidence (Bennequin, 2019).\u003c/p\u003e\n\u003cp\u003eIdentifying that MAML only learns an initial model since its updating rule is fixed to a classic gradient descent method, Jamal and Qi (2019) argued that such a biased initial model may not be well generalizable to an unseen task deviating from meta-training tasks. Therefore, they proposed a Task-Agnostic Meta-Learning (TAML) algorithm for few-shot learning. To solve the central issue of MAML, the discrepancy of future tasks and training task, they imposed an unbiased task-agnostic prior on the initial model by avoiding it from over-performing at some tasks; this allowed the meta-learning to achieve a more competitive update rule.\u003c/p\u003e\n\u003cp\u003eThere are two novel paradigms of TAML, one entropy-based and one inequality-minimization measures based. While the entropy-based means is only amenable to discrete labels and thus not as sound for regressing and reinforcement learning, the inequality-minimization-measures-based means, using measures like Theil Index, Atkinson Index, and Gini-Coefficient, is applicable to wider settings. However, experimentation exhibits that the entropy-based approach performs better than the inequality-based approach. Compared with MAML, both an entropy-based TAML and a general inequality-minimization TAML applicable to more ubiquitous scenarios are presented. This implies that TMAL could consistently outperform existing meta-learning algorithms on few-shot learning.\u003c/p\u003e\n\u003cp\u003eThough recent deep learning algorithms, relying on the ability to apply gradient-based optimization, have made considerable breakthroughs, there’s a persistent challenge on one-shot learning, a special case of few-shot learning. In the conventional approach of deep neural networks, a model must “forget” its trained parameter as to adequately incorporate the new information. However, Santoro \u003cem\u003eet al.\u003c/em\u003e (2016) claim that neural net workswith a memory capacity provide a promising approach to meta-learning in deep networks.\u003c/p\u003e\n\u003cp\u003eThey demonstrated that a highly capable memory-augmented neural network’s (MANN) capability of meta-learning in tasks that carry significant shot- and long-term memory demands. Also, they showed that the ability to slowly learn an abstract method for obtaining useful representations of raw data (via gradient-descent) and the ability to rapidly bind new information after a single presentation (via an external memory module) are combined in their approach. This again implies that such one-shot learning with memory-augmented neural networks can be effective for one-shot learning.\u003c/p\u003e\n\u003cp\u003eExperimental results on Omniglot classification and regression tasks confirmed that this model compares well to human baseline and other models like feedforward and LSTM. Such superior performance present even when only sparse training data was available, as a result of gradual, incremental learning in conjunction with a more flexible memory resource.\u003c/p\u003e\n\u003cp\u003eAnother extreme counterpart of few-shot learning, Zero-Shot Learning (ZSL) approaches can tackle challenging circumstances in which new categories appear after the learning stage. It is inspired by the human ability to identify new objects by merely reading a description of it and relating them with previously learned concepts, and it is inherently a two-stage process. Romera-Paredes \u003cem\u003eet al.\u003c/em\u003e (2017) studied a framework that is capable of integrating both stages and, based on that, proposed a ZSL approach that can perform efficiently both on training and inference stages, as well as extremely simple to implement – one line of code for each stage. It models the relationships between features, attributes, and classes as a two linear layers network, where the weights of the top layer are not learned but are given by the environment. This method is aptly named Embarrassing Simple ZSL (ESZSL). Also, they provide an analysis of the bound of the generalization error of this approach.\u003c/p\u003e\n\u003cp\u003eBoth synthetic experiments and real data experiments show that ESZSL significantly outperforms state-of-the-art algorithms like DAP and ZSRwUA. A modified version, ESZSL All Signatures (ESZSL-AS), performs even better in some cases. This comports with the author’s claim that this method should be robust to attributes having different discriminative capabilities for characterizing the classes.\u003c/p\u003e\n\u003ch2 id=\"conclusion\"\u003e\n \u003cspan\u003eConclusion\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#conclusion\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cp\u003eThis survey shows three exemplary ways at which meta-learning embodies itself. From Model Agnostic Meta-Learning to Meta-Reinforcement Learning and Few-Shot Learning, each category contains varied implementations. The survey presented the iteration and evolution of algorithm series like MAML—Alpha-MAML—MAML++, the attempts of different meta-learning applications, like RL in robotics, image recognition, and simulation, as well as the specialization of algorithms, like few-shot learning as a few-sample adaption of supervised meta-learning. Along with other recent meta-learning works, they uncovered that exploiting meta-knowledge can be reached in many directions; in other words, there is a wide spectrum of approaches yet to be explored.\u003c/p\u003e\n\u003cp\u003eThe No Free Lunch principle dictated that an optimization algorithm cannot outperform even the base-line model for all possible problems (Wolpert \u0026amp; Macready, 1997); however, meta-learning optimizations do not apply to its settings. The fundamental structure of meta-learning is distinctive from learning at the base level, considering the presence of meta-knowledge. It more resembles the human learning structure, where a large amount of meta-information can be stored. As Giraud-Carrier and Provost claimed, meta-learning offers a natural alternative to constructing a general-purpose learning algorithm, and though NFL may apply to the meta-level, the cost of our free lunch would be theoretically reasonable (Giraud-Carrier \u0026amp; Provost, 2005). Keeping an open view to novel means towards the exploitation of meta-knowledge can perhaps allow better insights. We have encountered quirky methods like learning gradient descent by gradient descent and learning with zero-shot in one line of code, so, if not expecting overly simple hacks, we should anticipate an epiphany – both in machines’ and our minds.\u003c/p\u003e\n\u003ch2 id=\"references\"\u003e\n \u003cspan\u003eReferences\u003c/span\u003e\n \u003ca class=\"heading-anchor\" href=\"#references\" aria-label=\"Link to this section\"\u003e\u003c/a\u003e\n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003e\n\u003cp\u003eAndrychowicz, M., Denil, M., Gomez, S., Hoffman, M. W., Pfau, D., Schaul, T.\n, Shillingford, B., \u0026amp; de Freitas, N. (2016). Learning to learn by gradient descent by gradient descent. \u003cem\u003eArXiv:1606.04474 [Cs]\u003c/em\u003e. \n\n\n\n\n\n\u003ca href=\"http://arxiv.org/abs/1606.04474\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttp://arxiv.org/abs/1606.04474\u003c/a\u003e\n\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eAntoniou, A., Edwards, H., \u0026amp; Storkey, A. (2019). How to train your MAML. \u003cem\u003eArXiv:1810.09502 [Cs, Stat]\u003c/em\u003e. \n\n\n\n\n\n\u003ca href=\"http://arxiv.org/abs/1810.09502\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttp://arxiv.org/abs/1810.09502\u003c/a\u003e\n\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eBehl, H. S., Baydin, A. G., \u0026amp; Torr, P. H. S. (2019). Alpha MAML: Adaptive\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eModel-Agnostic Meta-Learning. \u003cem\u003eArXiv:1905.07435 [Cs, Stat]\u003c/em\u003e. http://arxiv.\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eorg/abs/1905.07435\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eBennequin, E. (2019). Meta-learning algorithms for Few-Shot Computer Vision.\n\u003cem\u003eArXiv:1909.13579 [Cs]\u003c/em\u003e. \n\n\n\n\n\n\u003ca href=\"http://arxiv.org/abs/1909.13579\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttp://arxiv.org/abs/1909.13579\u003c/a\u003e\n\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eBhatt, N., Thakkar, A., \u0026amp; Ganatra, A. (2012). \u003cem\u003eA Survey \u0026amp; Current Research\nChallenges in Meta Learning Approaches based on Dataset Characteristics\u003c/em\u003e. \u003cem\u003e2\u003c/em\u003e(1), 9.\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eBiggs, J. B. (1985). The Role of Metalearning in Study Processes. \u003cem\u003eBritish\nJournal of Educational Psychology\u003c/em\u003e, \u003cem\u003e55\u003c/em\u003e(3), 185–212. \n\n\n\n\n\n\u003ca href=\"https://doi.org/10.1111/j.2044-8279.1985.tb02625.x\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttps://doi.org/10.1111/j.2044-8279.1985.tb02625.x\u003c/a\u003e\n\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eBoser, B. E., Guyon, I. M., \u0026amp; Vapnik, V. N. (1992). A training algorithm\nfor optimal margin classifiers. \u003cem\u003eProceedings of the Fifth Annual Workshop on Computational Learning Theory - COLT ’92\u003c/em\u003e, 144–152. \n\n\n\n\n\n\u003ca href=\"https://doi.org/10.1145/130385.130401\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttps://doi.org/10.1145/130385.130401\u003c/a\u003e\n\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eBrazdil, P., Giraud-Carrier, C., Soares, C., \u0026amp; Vilalta, R. (2009).\n\u003cem\u003eMetalearning: Applications to Data Mining\u003c/em\u003e. Springer Berlin Heidelberg. \n\n\n\n\n\n\u003ca href=\"https://doi.org/10.1007/978-3-540-73263-1\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttps://doi.org/10.1007/978-3-540-73263-1\u003c/a\u003e\n\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eBreiman, L. (1996). Bagging Predictors. \u003cem\u003eMachine Learning\u003c/em\u003e, \u003cem\u003e24\u003c/em\u003e(2),\n123–140. \n\n\n\n\n\n\u003ca href=\"https://doi.org/10.1023/A:1018054314350\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttps://doi.org/10.1023/A:1018054314350\u003c/a\u003e\n\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eBreiman, L. (2001). \u003cem\u003eRandom Forests\u003c/em\u003e. Kluwer Academic Publishers.\n\n\n\n\n\n\n\u003ca href=\"https://doi.org/10.1023/A:1010933404324\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttps://doi.org/10.1023/A:1010933404324\u003c/a\u003e\n\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eDu, Y., \u0026amp; Narasimhan, K. (2019). Task-Agnostic Dynamics Priors for Deep\nReinforcement Learning. \u003cem\u003eArXiv:1905.04819 [Cs, Stat]\u003c/em\u003e. \n\n\n\n\n\n\u003ca href=\"http://arxiv.org/abs/1905.04819\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttp://arxiv.org/abs/1905.04819\u003c/a\u003e\n\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eFinn, C., Abbeel, P., \u0026amp; Levine, S. (2017). Model-Agnostic Meta-Learning for\nFast Adaptation of Deep Networks. \u003cem\u003eArXiv:1703.03400 [Cs]\u003c/em\u003e. \n\n\n\n\n\n\u003ca href=\"http://arxiv.org/abs/1703.03400\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttp://arxiv.org/abs/1703.03400\u003c/a\u003e\n\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eFinn, C., Yu, T., Fu, J., Abbeel, P., \u0026amp; Levine, S. (2017). Generalizing\nSkills with Semi-Supervised Reinforcement Learning. \u003cem\u003eArXiv:1612.00429 [Cs]\u003c/em\u003e. \n\n\n\n\n\n\u003ca href=\"http://arxiv.org/abs/1612.00429\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttp://arxiv.org/abs/1612.00429\u003c/a\u003e\n\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eFreund, Y., \u0026amp; Schapire, R. E. (1997). A Decision-Theoretic Generalization\nof On-Line Learning and an Application to Boosting. \u003cem\u003eJournal of Computer and System Sciences\u003c/em\u003e, \u003cem\u003e55\u003c/em\u003e(1), 119–139. \n\n\n\n\n\n\u003ca href=\"https://doi.org/10.1006/jcss.1997.1504\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttps://doi.org/10.1006/jcss.1997.1504\u003c/a\u003e\n\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eGiraud-Carrier, C., \u0026amp; Provost, F. (2005). Toward a justification of\nmeta-learning: Is the no free lunch theorem a show-stopper? \u003cem\u003eProceedings of the ICML-2005 Workshop on Meta-Learning\u003c/em\u003e.\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eHochreiter, S., Younger, A. S., \u0026amp; Conwell, P. R. (2001). Learning to Learn\nUsing Gradient Descent. In G. Dorffner, H. Bischof, \u0026amp; K. Hornik (Eds.), \u003cem\u003eArtificial Neural Networks—ICANN 2001\u003c/em\u003e (Vol. 2130, pp. 87–94). Springer Berlin Heidelberg. \n\n\n\n\n\n\u003ca href=\"https://doi.org/10.1007/3-540-44668-0_13\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttps://doi.org/10.1007/3-540-44668-0_13\u003c/a\u003e\n\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eHsu, K., Levine, S., \u0026amp; Finn, C. (2019). Unsupervised Learning via\nMeta-Learning. \u003cem\u003eArXiv:1810.02334 [Cs, Stat]\u003c/em\u003e. \n\n\n\n\n\n\u003ca href=\"http://arxiv.org/abs/1810.02334\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttp://arxiv.org/abs/1810.02334\u003c/a\u003e\n\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eJamal, M. A., \u0026amp; Qi, G.-J. (2019). Task Agnostic Meta-Learning for Few-Shot\nLearning. \u003cem\u003e2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)\u003c/em\u003e, 11711–11719. \n\n\n\n\n\n\u003ca href=\"https://doi.org/10.1109/CVPR.2019.01199\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttps://doi.org/10.1109/CVPR.2019.01199\u003c/a\u003e\n\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eJensen, F. V. (1996). \u003cem\u003eIntroduction to Bayesian Networks\u003c/em\u003e (1st ed.).\nSpringer-Verlag.\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eKrizhevsky, A., Sutskever, I., \u0026amp; Hinton, G. E. (2017). ImageNet\nclassification with deep convolutional neural networks. \u003cem\u003eCommunications of the ACM\u003c/em\u003e, \u003cem\u003e60\u003c/em\u003e(6), 84–90. \n\n\n\n\n\n\u003ca href=\"https://doi.org/10.1145/3065386\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttps://doi.org/10.1145/3065386\u003c/a\u003e\n\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eLan, L., Li, Z., Guan, X., \u0026amp; Wang, P. (2019). Meta Reinforcement Learning\nwith Task Embedding and Shared Policy. \u003cem\u003eArXiv:1905.06527 [Cs, Stat]\u003c/em\u003e. \n\n\n\n\n\n\u003ca href=\"http://arxiv.org/abs/1905.06527\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttp://arxiv.org/abs/1905.06527\u003c/a\u003e\n\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eLemke, C., Budka, M., \u0026amp; Gabrys, B. (2015). Metalearning: A survey of trends\nand technologies. \u003cem\u003eArtificial Intelligence Review\u003c/em\u003e, \u003cem\u003e44\u003c/em\u003e(1), 117–130. \n\n\n\n\n\n\u003ca href=\"https://doi.org/10.1007/s10462-013-9406-y\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttps://doi.org/10.1007/s10462-013-9406-y\u003c/a\u003e\n\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eMendonca, R., Gupta, A., Kralev, R., Abbeel, P., Levine, S., \u0026amp; Finn, C.\n(2019). Guided Meta-Policy Search. \u003cem\u003eArXiv:1904.00956 [Cs, Stat]\u003c/em\u003e. \n\n\n\n\n\n\u003ca href=\"http://arxiv.org/abs/1904.00956\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttp://arxiv.org/abs/1904.00956\u003c/a\u003e\n\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eMucherino, A., Papajorgji, P. J., \u0026amp; Pardalos, P. M. (2009). K-Nearest\nNeighbor Classification. In A. Mucherino, P. J. Papajorgji, \u0026amp; P. M. Pardalos (Eds.), \u003cem\u003eData Mining in Agriculture\u003c/em\u003e (pp. 83–106). Springer. \n\n\n\n\n\n\u003ca href=\"https://doi.org/10.1007/978-0-387-88615-2_4\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttps://doi.org/10.1007/978-0-387-88615-2_4\u003c/a\u003e\n\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eMurphy, K. P. (2012). \u003cem\u003eMachine learning: A probabilistic perspective\u003c/em\u003e. MIT\nPress.\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eOpenAI, Akkaya, I., Andrychowicz, M., Chociej, M., Litwin, M., McGrew, B.,\nPetron, A., Paino, A., Plappert, M., Powell, G., Ribas, R., Schneider, J., Tezak, N., Tworek, J., Welinder, P., Weng, L., Yuan, Q., Zaremba, W., \u0026amp; Zhang, L. (2019). Solving Rubik’s Cube with a Robot Hand. \u003cem\u003eArXiv:1910.07113 [Cs, Stat]\u003c/em\u003e. \n\n\n\n\n\n\u003ca href=\"http://arxiv.org/abs/1910.07113\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttp://arxiv.org/abs/1910.07113\u003c/a\u003e\n\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eQuinlan, J. R. (1986). Induction of decision trees. \u003cem\u003eMachine Learning\u003c/em\u003e, \u003cem\u003e1\u003c/em\u003e\n(1), 81–106. \n\n\n\n\n\n\u003ca href=\"https://doi.org/10.1007/BF00116251\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttps://doi.org/10.1007/BF00116251\u003c/a\u003e\n\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eRice, J. R. (1976). The Algorithm Selection Problem. In \u003cem\u003eAdvances in\nComputers\u003c/em\u003e (Vol. 15, pp. 65–118). Elsevier. \n\n\n\n\n\n\u003ca href=\"https://doi.org/10.1016/S0065-2458%2808%2960520-3\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttps://doi.org/10.1016/S0065-2458(08)60520-3\u003c/a\u003e\n\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eRomera-Paredes, B., \u0026amp; Torr, P. H. S. (2017). An Embarrassingly Simple\nApproach to Zero-Shot Learning. In R. S. Feris, C. Lampert, \u0026amp; D. Parikh (Eds.), \u003cem\u003eVisual Attributes\u003c/em\u003e (pp. 11–30). Springer International Publishing. \n\n\n\n\n\n\u003ca href=\"https://doi.org/10.1007/978-3-319-50077-5_2\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttps://doi.org/10.1007/978-3-319-50077-5_2\u003c/a\u003e\n\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eSantoro, A., Bartunov, S., Botvinick, M., Wierstra, D., \u0026amp; Lillicrap, T.\n(2016). One-shot Learning with Memory-Augmented Neural Networks. \u003cem\u003eArXiv:1605.06065 [Cs]\u003c/em\u003e. \n\n\n\n\n\n\u003ca href=\"http://arxiv.org/abs/1605.06065\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttp://arxiv.org/abs/1605.06065\u003c/a\u003e\n\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eSilver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., Driessche, G.\nvan den, Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., Dieleman, S., Grewe, D., Nham, J., Kalchbrenner, N., Sutskever, I., Lillicrap, T., Leach, M., Kavukcuoglu, K., Graepel, T., \u0026amp; Hassabis, D. (2016). Mastering the game of Go with deep neural networks and tree search. \u003cem\u003eNature\u003c/em\u003e, \u003cem\u003e529\u003c/em\u003e, 484–503.\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eSutton, R. (2019, March 13). \u003cem\u003eThe Bitter Lesson\u003c/em\u003e. http://incompleteideas.\nnet/IncIdeas/BitterLesson.html\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eVanschoren, J. (2018). Meta-Learning: A Survey. *ArXiv:1810.03548 [Cs, Stat]\n*. \n\n\n\n\n\n\u003ca href=\"http://arxiv.org/abs/1810.03548\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttp://arxiv.org/abs/1810.03548\u003c/a\u003e\n\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eVilalta, R., \u0026amp; Drissi, Y. (2002). \u003cem\u003eA Perspective View and Survey of\nMeta-Learning\u003c/em\u003e. 20.\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eWeng, L. (2019, June 23). \u003cem\u003eMeta Reinforcement Learning\u003c/em\u003e. Lil’Log.\n\n\n\n\n\n\n\u003ca href=\"https://lilianweng.github.io/2019/06/23/meta-reinforcement-learning.html\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttps://lilianweng.github.io/2019/06/23/meta-reinforcement-learning.html\u003c/a\u003e\n\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eWolpert, D. H., \u0026amp; Macready, W. G. (1997). No free lunch theorems for\noptimization. \u003cem\u003eIEEE Transactions on Evolutionary Computation\u003c/em\u003e, \u003cem\u003e1\u003c/em\u003e(1), 67–82. \n\n\n\n\n\n\u003ca href=\"https://doi.org/10.1109/4235.585893\" target=\"_blank\" rel=\"external noopener noreferrer\"\u003ehttps://doi.org/10.1109/4235.585893\u003c/a\u003e\n\u003c/p\u003e\n\u003c/li\u003e\n\u003c/ul\u003e\n","date_modified":"2026-05-30T00:37:17-07:00","date_published":"2019-06-14T00:00:00Z","id":"https://www.xiyan.dev/writings/2019/06/14/a-survey-of-recent-meta-learning-perspectives-approaches-and-applications/","summary":"A high-school-era survey of meta-learning, or learning to learn, covering Model-Agnostic Meta-Learning, Meta-Reinforcement Learning, and few-shot methods alongside their mechanisms, applications, and performance.","tags":["Meta-learning","Literature Review"],"title":"A Survey of Recent Meta-Learning Perspectives, Approaches, and Applications","url":"https://www.xiyan.dev/writings/2019/06/14/a-survey-of-recent-meta-learning-perspectives-approaches-and-applications/"}],"language":"en-us","title":"Xiyan's Blog","version":"https://jsonfeed.org/version/1.1"}