Skip to content

Latest commit

 

History

History
361 lines (331 loc) · 13.2 KB

NOTES.md

File metadata and controls

361 lines (331 loc) · 13.2 KB

some notes

  • bundler plugins
    • two phases: generate config, generate build
  • lavamoat internals
    • requireRelativeWithContext
      • passed directly to moduleInitializer (untrusted code)
      • calls requireRelative with parentModule context added
    • requireRelative
      • translates the "requestedName" (eg relative path) to a moduleId (webpack doesnt need this, it rewrites the source to use the moduleId directly)
      • handles weird browserify-specific recursive lookups (eg buffer, timers)
      • calls internalRequire
      • uses the module config's protectExportsRequireTime
    • internalRequire:
      • instantiates the module in the specified context with
  • known / potential issues
    • sneaky setting of packageName by dir hacks?
    • magicCopy doesnt walk the full prototype graph
    • perf overhead
    • doesnt solve architectural weaknesses
    • doesnt enforce disabling scripts (yet!)

LA audit kickoff todos

  • clarify requireFns
  • audit cache, looks broken
  • transfer sesify-tofu
  • move as much code into ses as possible
  • remove alternate export protection strategies (underdeveloped)
  • comment shit you cowboy

another list of todos

  • isEntry based on packageName

  • remove modulePath

  • unify on <entry> or <root>

  • remove providedEndowments

  • enforce deps whitelist in config

  • devex would be improved if --config was constant and --writeAutoConfig used the --config path as destination

  • unit test kernel components

  • breakout kernel / kernel building utils

  • unify on depMap (?)

  • unify on kernel/prelude/runtime

  • test sneaky setting of packageName by dir hacks

  • module mappings

    • requestedName -> moduleId
    • moduleId -> { packageName, path }

  • idea: maybe encode dep graph (and cache) as weakmap pointers so they disappear as require ref is dropped

  • document browserify usage

  • using --writeAutoConfig should create a valid build using the generated config

exportsProtection strategies

  • harden strategy
    • tests
  • fresh eval/instantion doesnt require magicCopy

summary of components

  • Config

  • Containment

    • can happen less granularly than modules
      • in addition to granularity of modules
    • module instance per path? purity?
      • no, it explodes
    • if config per-path we need to split eval and endowments
      • if you want special config, do higher level containment
    • how to correctly specify the global object
    • how to do attenuations
      • specify module/global replacements
      • maybe add a config for common attenuations

recent notes + todo

  • importer can decorate: common
  • importer can see late sets: uncommonn

SES

  • thoughts

    • worried im blocked by the typeof xyz erroring issue
    • can work around globalThis.Object === Object
    • need to summarize endowments-sloppyGlobals-globalThis requirements

  • ok a plan of action

    • use compartments
    • remove sloppyGlobals
    • define getters/setters on readable/writable globals

  • autoconfig / global detection does not support writes

    • re-examine tofu
    • otherwise consider whats needed to detect

  • evaluate current proposal (fixed in SES)

    • await feedback
    • examine for differences in using getters on endowments
    • test in metamask
      • Symbol.iterator, Symbol.asyncIterator, Symbol.toStringTag

  • document and diagram scope

    • endowments
    • realm.global
    • cjs module source

  • endowment ref tree generation is kinda broken

    • how to handle deep sets?

  • Function constructor globalThis hack (?)

  • also interested in doing deeper dep analysis for dep dashboard

    • draw lines for global writes to global reads

debug bundle sesh in metamask

  • ?? -- in sentry deps, mod.require is undefined. module.require appears in node maybe fine, as we already use this in browserify and its not supported

    • "obs-store" doesnt have localStorage access despite use?
    • setTimeout not whitelisted for package "process"
      • problem: packageName is "_process"
    • ui deps maps "react-dom" to "react-dom", not moduleId (why?)
      • external deps (uideps bundle)
    • metamask minifying the whole bundle, including kernel?

  • sesify

    • allow deletes for global writes
    • must support globalThis getter via Function
    • loading "buffer" fails - seems not added to parent modules deps map (witnessed in bn.js) why: node_modules/ethjs-abi/node_modules/bn.js/lib/bn.js | grep "'buf' + 'fer'" need to upgrade to a proper error
    • drop console warns
    • debugging: label package + file name
    • globals
      • global writes
        • feature
        • test
        • autoconfig
      • inspect globals
        • should check for anything
          • "Blob"
          • "regeneratorRuntime"
        • fix endowment generation
          • Blob.prototype is undefined
    • workaround for assigning to proto
    • endowments
      • make sure generateEndowments works correctly with globalStore
        • seeing an error with regeneratorRuntime.mark in eth-json-rpc-middleware / json-rpc-engine
        • cant call defineProperty on a string (location.href.indexOf)
    • options
      • conflicts when autogen + config is specified
      • use autogen config in output
      • investigate "unfrozen" reccomendation reasoning
    • DevEx
      • seems to break on live reload
        • doesnt seem to get new config on reload?
        • reload on config change only works once?

  • babel-thing (obsolete due to fix in SES)

    • handle frozen prototype writes to next (iterator)
    • length of array is writable but not configurable, so defining 'length' fails (in bignumber.js)

  • metamask

    • "content-hash" current version contains reserved word "package"
    • sentry does weird shit, maybe remove it for now
    • inpage, cleanContextForImports called before var dec? (due to error?)
    • transforming ui with babel-thing
    • shouldnt need regeneratorRuntime ?
      • getting it in eth-json-rpc-middleware / json-rpc-engine
    • aes-js (old ver) modifies Array.prototype dep paths: eth-hd-keyring eth-simple-keyring gaba ethereumjs-wallet aes-js

older bundleEntryForModule

todo

  • shared instances of modules
    • revert the seperation of eval / global injection
  • make config like agoric prototype
    • config is json
    • global-grabbing engine needs to move to sesify prelude
  • update SES
    • some issue with prelude or SES kernel running in strict mode
  • shared instances of modules
    • revert the seperation of eval / global injection
    • update cache key generator
  • mark question: while this is container global, self is undefined
  • fix globalRefs
    • fix objCheckSelf
    • fix objCheckGlobal
  • new config
    • make config like agoric prototype
    • config is json
    • config -> endowments in sesify prelude
    • need tests that generate config then use it
    • cleanup old config generation
    • get packageName from modules stream
    • allow easy override of configuration
    • no longer de-duping overlapping namespaces? (needs test)
  • config advanced
    • execution environment options
      • autogen config from "alt environment heuristics"
      • execute unfrozen in fresh realm
    • defensibility/hardening options
      • magic copy
      • kowtow
      • return raw if same package (dont defense)
      • @agoric/harden
        • ? metamask error hardening proto chain
    • enforce configuration
      • fail at buildtime if deps violation
      • enforce globals
    • move SES config into sesify config
  • mystery bugs
    • this.Object gets transformed to undefined.Object in mm
  • cleanup prelude
  • sesify metamask
    • autogen config
    • setup build sys
    • debug boot
    • debug runtime
    • gulp task for autogen
  • improve pluginOpts
    • config vs sesifyConfig
    • if autoconfig, use that config
  • question
    • sesify with unfrozen realm
      • should be on Realm.evaluate, verify
    • how to create a copy of a fn class

if autogen config

  • allow browserPack to pause stream until config is generated
  • generate config then unpause browserPack
  • back pressure could cause a dead lock (?)

sesify prelude/kernel

  • pass custom endowments at require time

  • pass custom endowments at config time

    • get config into bundle
    • lookup config by module id / dep path
    • [?] how to deal with entry point name if entries specified by id / multiple entry points

  • include SES in prelude

  • share realm for all files in module?

  • make global module config as well

  • allow some sort of global realm sharing

  • set custom prelude in browserify via plugin

    • works but sometimes breaks things...
    • plugin without breaking things via b.reset()?

  • need to not break sourcemaps

    • good enough for now
    • handle module names with @xyz/abc format

  • lockdown everything thats passed to module initializer

    • wrap newRequire, etc
    • remove excessive + dangerous moduleInitializer args
      • investigate why corejs was using arguments[4] and see if others are too

  • cleanup prelude

  • is global caching safe? (no)

  • try using the frozen realm + container architecture

  • battletest via metamask

    • background boot works : )
    • sent first tx for background-only sesified
    • contentscript doesnt?
    • find sane default endowments

  • support granular config

    • actually expose api from granular config
    • ensure we keep the "this" context, esp for deepGets
    • ensure we dont break Constructors with our "this" fix

  • browserify insertGlobal is ruining the parsing of properties on global

  • sourcemaps

    • needs to be able to compose over existing sourcemaps
    • needs to work when there are no existing sourcemaps
    • config to specify inline or file
    • config to dump map somewhere file
    • [?] ahhhhh nested inline sourcemaps?? not my problem??

  • (external) allow less restrictive sandboxing modes (prototype.toString())

  • (external) closer control over global? pass in "window" such that (window.Object === Object)

  • [?] browserify the prelude

tofu parser

  • mvp
    • analyze required files for platform API usage
    • use this to spit out a sesify config file (or something)
    • get dependency info
    • use generated config
  • not terrible
    • more granular autogen config
      • detect API usage on global
      • dont pass window if no property accessed
      • granularity on certain apis, e.g. document
      • raise platform api granularity to common denominator (e.g. dedupe "location" and "location.href"), including defaultGlobals
      • [?] maybe limit granularity to actual platform API surface (e.g. not "location.href.indexOf")
      • browserify insertGlobal is ruining the parsing of properties on global
        • bc declaring the global object and passing it into a closure causes acorn-globals to ignore the uses of the global var
    • user config defaultGlobals
    • [?] location and document.location is redundant
    • location and location.href trigers page reload < !!! wow ouch !!! >
    • easy user override
      • likely need revDeps pointers at run time
    • use SES.confine instead of realm.evaluate
    • update ses

audit

  • basic safety review
  • LA audit

devx

  • use autogen config if set to generate ?
  • cli support (?)
    • config gen
    • config read
  • lots of config noise due to console, setTimeout
  • sourcemaps
    • metamask sourcemaps are already a mess
    • then sesify sourcemaps get a bit worse

idea icebox

  • permissions as higher abstractions (network, persistence, DOM)
  • permissions sorted by risk (?)

the big ones

  • autogen granularity
  • sourcemaps
  • do call with agoric/MM
  • (external) SES prototype.toString etc
  • perf

make perf better

  • reduce instantiations
    • allow module cache under certain conditions
      • cant harden exports?
    • lazy instantiation via "universal proxy"
      • my guess: we use most things on boot
  • improve instantiation perf
    • transform src with endowments injection (build time slow down)

dangers of module cache

  • mutating the exports
  • cant be done if it includes unhardenables (?)

cli testing

eval in sesify bundle

echo 'console.log(self.process === process)' | browserify - --detect-globals false --no-builtins -p [ './src/index.js' --sesifyConfig '{"resources":{"<root>":{"globals":{"console":true,"process":true}}}}' ] | node

eval in ses

node -p "try { require('ses').makeSESRootRealm().evaluate('const x = {}; x.hasOwnProperty = ()=>{}') } catch (err) { console.log(err.message) }"

npm bug workaround

npm unlink sesify && npm i && npm link sesify