Module refinery.lib.scripts.ps1.analysis

Static-analysis substrate for PowerShell deobfuscation. Transforms query a shared, computed model of the program here instead of each re-deriving scope, binding, effect, and liveness facts on their own.

The foundation is refinery.lib.scripts.ps1.analysis.model, a semantic model of scopes and resolved variable bindings, and refinery.lib.scripts.ps1.analysis.callgraph, which records what a command name denotes and who reaches it. On top of both, refinery.lib.scripts.ps1.analysis.effects decides what evaluating a node does, whether it can raise, and — through the call graph — where the value a body writes to the output stream is finally read. Later layers (control-flow graphs, interprocedural summaries) attach behind the same representation-agnostic surface.

"""
Static-analysis substrate for PowerShell deobfuscation. Transforms query a shared, computed model of
the program here instead of each re-deriving scope, binding, effect, and liveness facts on their
own.

The foundation is `model`, a semantic model of scopes and resolved variable bindings, and
`callgraph`, which records what a command name denotes and who reaches it. On top of both, `effects`
decides what evaluating a node does, whether it can raise, and — through the call graph — where the
value a body writes to the output stream is finally read. Later layers (control-flow graphs,
interprocedural summaries) attach behind the same representation-agnostic surface.
"""
from __future__ import annotations

from refinery.lib.scripts.ps1.analysis.cache import Ps1ModelCache, model_cache
from refinery.lib.scripts.ps1.analysis.effects import (
    StatementEffect,
    is_side_effect_free,
    statement_effect,
)
from refinery.lib.scripts.ps1.analysis.model import (
    Binding,
    Ps1OccurrenceRole,
    Ps1SemanticModel,
    Scope,
    ScopeKind,
    build_semantic_model,
    declares_binding,
    is_assignment_write_target,
    is_substitutable_position,
    is_write_occurrence,
    occurrence_role,
    replaces_value,
)

__all__ = [
    'Binding',
    'Ps1ModelCache',
    'Ps1OccurrenceRole',
    'Ps1SemanticModel',
    'Scope',
    'ScopeKind',
    'StatementEffect',
    'build_semantic_model',
    'declares_binding',
    'is_assignment_write_target',
    'is_side_effect_free',
    'is_substitutable_position',
    'is_write_occurrence',
    'model_cache',
    'occurrence_role',
    'replaces_value',
    'statement_effect',
]

Sub-modules

refinery.lib.scripts.ps1.analysis.arguments

Which slots of a .NET call the callee writes through …

refinery.lib.scripts.ps1.analysis.blocks

Where a PowerShell script block runs: at what point, in whose scope, and how many times …

refinery.lib.scripts.ps1.analysis.cache

A per-run cache of the PowerShell analysis models. The deobfuscation pipeline builds one cache over the script being transformed and shares it across …

refinery.lib.scripts.ps1.analysis.callgraph

The call graph of one PowerShell script: which function definitions a command name denotes, which invocations reach that name, and whether the two …

refinery.lib.scripts.ps1.analysis.cfg

PowerShell's contribution to the shared control-flow substrate: which node types are which control-flow shape, and where the parts of each are stored …

refinery.lib.scripts.ps1.analysis.commands

What command a name denotes at a point in one PowerShell script …

refinery.lib.scripts.ps1.analysis.dataflow

Which write a PowerShell variable read observes …

refinery.lib.scripts.ps1.analysis.dominance

Dominance over the per-body control-flow graphs of one PowerShell script …

refinery.lib.scripts.ps1.analysis.effects

The effect layer of the PowerShell analysis substrate: whether evaluating a node produces an observable side effect, and what a standalone statement …

refinery.lib.scripts.ps1.analysis.faults

Where a terminating error raised at a point in a PowerShell script goes …

refinery.lib.scripts.ps1.analysis.model

A semantic model for PowerShell: a tree of scopes with resolved variable bindings and def/use sets, computed once over an AST and then queried by the …

refinery.lib.scripts.ps1.analysis.mutation

What a call leaves in the slot it writes through …

refinery.lib.scripts.ps1.analysis.naming

The names a script addresses as strings rather than as variables …

refinery.lib.scripts.ps1.analysis.opaque

The writes no reading of one script attributes to a name …

refinery.lib.scripts.ps1.analysis.separator

What a collection is written with between its elements where PowerShell coerces one to a String …

refinery.lib.scripts.ps1.analysis.values

What a PowerShell expression evaluates to and what type that value carries: the value when the source pins it, the .NET type when the static surface …

refinery.lib.scripts.ps1.analysis.variable_types

The .NET type a variable carries where it is read …

refinery.lib.scripts.ps1.analysis.world

The closed-world model of the PowerShell analysis substrate: whether the script leaves the .NET type system and the command table intact, so that a …

refinery.lib.scripts.ps1.analysis.worldflow

The flow-sensitive reading of the closed-world model: whether the type world is closed at one particular read, rather than anywhere the script runs …

Functions

def build_semantic_model(root)

Build the Ps1SemanticModel for a parsed PowerShell script.

Expand source code Browse git

def build_semantic_model(root: Ps1Script) -> Ps1SemanticModel:
    """
    Build the `Ps1SemanticModel` for a parsed PowerShell script.
    """
    return Ps1SemanticModel(root)

def declares_binding(var)

Whether the occurrence brings the binding into existence in the scope it resolves to.

Every write that installs a value does, except a reference: PowerShell resolves [ref]$n by ordinary lookup and creates nothing, so filing one as a declaration invents a local binding in whatever body the reference is written in and hides the outer one the callee actually stores through. A write that reaches through the value declares nothing for the same reason: it needs a value to reach into, so the binding it names already exists wherever it exists.

Expand source code Browse git

def declares_binding(var: Ps1Variable) -> bool:
    """
    Whether the occurrence brings the binding into existence in the scope it resolves to.

    Every write that installs a value does, except a reference: PowerShell resolves `[ref]$n` by
    ordinary lookup and creates nothing, so filing one as a declaration invents a local binding in
    whatever body the reference is written in and hides the outer one the callee actually stores
    through. A write that reaches *through* the value declares nothing for the same reason: it needs
    a value to reach into, so the binding it names already exists wherever it exists.
    """
    if is_reference_cast(var.parent):
        return False
    role = occurrence_role(var)
    return role.stores and not role.through

def is_assignment_write_target(var)

Whether var occupies the target position of an enclosing Ps1AssignmentExpression, including as an element of a multi-assignment Ps1ArrayLiteral target. Enclosing casts and parentheses are transparent.

A question about syntax rather than about role, which is why it is not derived from occurrence_role(): a foreach variable and a parameter replace the value exactly as a plain assignment target does and occupy no assignment at all.

Expand source code Browse git

def is_assignment_write_target(var: Ps1Variable) -> bool:
    """
    Whether `var` occupies the target position of an enclosing
    `refinery.lib.scripts.ps1.model.Ps1AssignmentExpression`, including as an element of a
    multi-assignment `refinery.lib.scripts.ps1.model.Ps1ArrayLiteral` target. Enclosing casts and
    parentheses are transparent.

    A question about syntax rather than about role, which is why it is not derived from
    `occurrence_role`: a `foreach` variable and a parameter replace the value exactly as a plain
    assignment target does and occupy no assignment at all.
    """
    return assignment_of(var) is not None

def is_side_effect_free(node, world)

Conservative check: return True only when evaluating node is guaranteed to produce no observable side effects beyond yielding a value. The refinery.lib.scripts.ps1.analysis.world decides whether a present-member grant may be trusted and whether a command name still denotes what the metadata says, each at the position of the node it is asked about: a world opened, or a name rebound, only by statements no path places before the node still answers for it.

A variable read is one of the few things that is free of its own accord, and $input is the exception the grant has to name: reading it advances an enumerator the statements below it read, so a statement whose only content is that read still changes what the next one writes.

Expand source code Browse git

def is_side_effect_free(node, world: Ps1WorldReach) -> bool:
    """
    Conservative check: return `True` only when evaluating `node` is guaranteed to produce no
    observable side effects beyond yielding a value. The `world` decides whether a present-member
    grant may be trusted and whether a command name still denotes what the metadata says, each at
    the position of the node it is asked about: a world opened, or a name rebound, only by
    statements no path places before the node still answers for it.

    A variable read is one of the few things that is free of its own accord, and `$input` is the
    exception the grant has to name: reading it advances an enumerator the statements below it read,
    so a statement whose only content is that read still changes what the next one writes.
    """
    if isinstance(node, _LITERAL_EXPRESSIONS):
        return True
    if isinstance(node, Ps1TypeExpression):
        return True
    if isinstance(node, Ps1Variable):
        return not _reads_the_pipeline_enumerator(node)
    if isinstance(node, Ps1ParenExpression):
        return node.expression is None or is_side_effect_free(node.expression, world)
    if isinstance(node, Ps1CastExpression):
        # A cast is a conversion the engine performs by calling into the target type, so it is a
        # present-type grant like any other: a remapped accelerator invalidates it, and a name the
        # metadata cannot resolve is not a type this analysis knows anything about. `Add-Type`, a
        # PowerShell `class` and `[Reflection.Assembly]::Load` all make such a name denote code —
        # PowerShell converts a string to it by running a constructor — so granting on the operand
        # alone deleted the call. Resolving is necessary here, not sufficient: a conversion to a
        # collected type can still run code (`[xml]$s` parses, and follows external DTDs), which
        # `_PURE_CAST_TYPES` is the eventual answer to. The world is read before either check
        # because it is a stored bool that can only veto, while both checks below walk.
        if not world.closed_at(node):
            return False
        if data.resolve_type(node.type_name) is None:
            return False
        return is_side_effect_free(node.operand, world)
    if isinstance(node, Ps1UnaryExpression):
        if node.operator in ('++', '--'):
            return False
        return is_side_effect_free(node.operand, world)
    if isinstance(node, Ps1BinaryExpression):
        # The regex operators write the automatic `$Matches`, which the statements after them read;
        # that is a store to shared engine state, not a value the expression merely yields, so the
        # operator has to be read and not just the operands. Deleting `$s -match 'p(.*)q'` left the
        # `$Matches[1]` that carries the payload reading an unset variable.
        if node.operator.lower() in _MATCH_OPERATORS:
            return False
        return is_side_effect_free(node.left, world) and is_side_effect_free(node.right, world)
    if isinstance(node, Ps1RangeExpression):
        return is_side_effect_free(node.start, world) and is_side_effect_free(node.end, world)
    if isinstance(node, Ps1ArrayLiteral):
        return all(is_side_effect_free(e, world) for e in node.elements)
    if isinstance(node, Ps1HashLiteral):
        return all(
            is_side_effect_free(key, world) and is_side_effect_free(value, world)
            for key, value in node.pairs
        )
    if isinstance(node, Ps1ArrayExpression):
        if len(node.body) == 1:
            stmt = node.body[0]
            if isinstance(stmt, Ps1ExpressionStatement) and stmt.expression is not None:
                return is_side_effect_free(stmt.expression, world)
        return len(node.body) == 0
    if isinstance(node, Ps1IndexExpression):
        # Indexing selects the `Item` member, so it is the bracket spelling of the member read
        # below and carries the same Extended Type System exposure.
        pure = is_side_effect_free(node.object, world) and is_side_effect_free(node.index, world)
        return _grant(pure, node, world)
    if isinstance(node, Ps1MemberAccess):
        # A read is side-effect free only when the object is pure to evaluate *and* selecting the
        # member runs no code. Returning the object's own purity was the fail-open shape this gate
        # replaces: it deleted `(Get-Process).Path`, an Extended Type System getter that shells out,
        # because the pipeline that produced the object was itself pure. A literal member name —
        # bare (`.Path`) or quoted (`.'Path'`) — names one member the gate can check; a computed
        # member name (`$x.$(...)`) leaves the selected member unknown, so a read through it can
        # never be proven pure however pure the name expression is.
        if not is_side_effect_free(node.object, world):
            return False
        member = get_member_name(node.member)
        if member is None:
            return False
        return _grant(_member_read_is_pure(node.object, member, world), node, world)
    if isinstance(node, Ps1InvokeMember):
        if not _arguments_are_pure(node.arguments, world):
            return False
        if node.access == Ps1AccessKind.STATIC:
            obj = node.object
            member = node.member
            # A computed or quoted member name (`[IO.Path]::$m()`, `[IO.Path]::'GetTempFileName'()`)
            # cannot be matched against the carve-outs, so the whole-type grant below must not fire
            # for it either — that is how an obfuscated call reaches the one writing member of an
            # otherwise pure type.
            if isinstance(obj, Ps1TypeExpression) and isinstance(member, str):
                # The type name is resolved through the collected metadata, not truncated, so every
                # spelling of a type lands on one canonical key, and a type the data does not
                # describe resolves to nothing and falls through to impure: the fail-closed default.
                resolved = data.resolve_type(obj.name)
                if resolved is not None:
                    type_key = resolved.generic_definition
                    key = (type_key, member.lower())
                    if key in _IMPURE_STATIC_METHODS:
                        return False
                    written = written_slots(
                        resolved, member, len(node.arguments), static=True)
                    if written.slots:
                        return _grant(
                            not _writes_shared_storage(written, node.arguments), node, world)
                    if not written.settled:
                        # The table names the member and no overload takes this many arguments, so
                        # 5.1 binds none and raises. Granting purity here would let the junk remover
                        # delete a statement that writes an error record and stops the pipeline.
                        return False
                    if _writes_through_out_parameter(obj.name, member, node.arguments):
                        return False
                    if type_key in _PURE_STATIC_METHOD_TYPES:
                        return _grant(True, node, world)
                    if key in _PURE_STATIC_METHODS:
                        return _grant(True, node, world)
        elif is_side_effect_free(node.object, world):
            member = node.member
            if isinstance(member, str) and member.lower() in _PURE_INSTANCE_METHODS:
                return _grant(True, node, world)
        return False
    if isinstance(node, Ps1CommandInvocation):
        if node.redirections:
            return False
        new_object = extract_new_object(node)
        if new_object is not None:
            if not world.may_trust_command_name_at('new-object', node):
                return False
            type_name, ctor_args = new_object
            resolved = data.resolve_type(type_name)
            if (
                resolved is not None
                and resolved.generic_definition in _PURE_STATIC_METHOD_TYPES
            ):
                return _grant(_arguments_are_pure(ctor_args, world), node, world)
            return False
        name = get_command_name(node)
        if name is None:
            return False
        name = name.lower()
        # A command a reachable statement may have rebound no longer surely runs what the metadata
        # describes, so its purity is not the built-in's. The gate is positional — the identity
        # twin of the `_grant` below — trusting the name only where no opener and no redefinition
        # of this very name can have run first.
        if not world.may_trust_command_name_at(name, node):
            return False
        # The pipeline set is checked through the same gate rather than after the plain one: three
        # of its four members are in both, so testing the plain set first would make the body check
        # below unreachable for `Where-Object`, `Select-Object` and `Sort-Object`.
        if name not in _PURE_CMDLETS and name not in _PURE_PIPELINE_CMDLETS:
            return False
        if not _command_arguments_are_pure(node, world):
            return False
        # Routed through `_grant` like every other grant. `may_trust_command_name_at` above refuses
        # a name a rebinding statement can reach; `_grant` here refuses a member the type world
        # does not hold at this node. The two read different floods — the name gate adds the
        # per-name redefinition flood the type axis never reads — so a name that passed the first
        # is still gated on the second, and narrowing either check back into the other would reopen
        # a fail-open hole with nothing in the path to catch it.
        if name in _PURE_PIPELINE_CMDLETS:
            return _grant(_command_body_is_pure(node, world), node, world)
        return _grant(True, node, world)
    if isinstance(node, Ps1Pipeline):
        return all(
            isinstance(el, Ps1PipelineElement)
            and not el.redirections
            and is_side_effect_free(el.expression, world)
            for el in node.elements
        )
    if isinstance(node, Ps1ExpandableString):
        return all(is_side_effect_free(p, world) for p in node.parts)
    return False

def is_substitutable_position(var)

Whether a value may be installed where var stands, replacing the occurrence.

This is not the complement of writing, and reading it off the role alone is what let two corruptions through. A splatted @p observes the value like any read, but it spreads an array over a command's parameters, and the array written in its place is one argument rather than several. A [ref]$n observes the value too, and the literal put in its place is a reference to nothing that the callee's store is silently lost through.

Expand source code Browse git

def is_substitutable_position(var: Ps1Variable) -> bool:
    """
    Whether a value may be installed where `var` stands, replacing the occurrence.

    This is not the complement of writing, and reading it off the role alone is what let two
    corruptions through. A splatted `@p` observes the value like any read, but it spreads an array
    over a command's parameters, and the array written in its place is one argument rather than
    several. A `[ref]$n` observes the value too, and the literal put in its place is a reference to
    nothing that the callee's store is silently lost through.
    """
    return occurrence_role(var).substitutable and not var.splatted

def is_write_occurrence(var)

Whether var occurs in a position that writes it: the target of an assignment (including a multi-assignment slot), the operand of a ++/-- update, the loop variable of a foreach, a parameter declaration, the operand of a [ref] cast, or a position an assignment stores through. Every other occurrence reads the variable.

A store through is one of these although it installs nothing. $x[0] = 'z' leaves the name bound to the object it was bound to, but a read below it observes a different value, and that is the whole of what a write is to the layer that orders reads against writes. Counting it a read instead is what forced Ps1VariableFlow to give up on every occurrence of the name.

Expand source code Browse git

def is_write_occurrence(var: Ps1Variable) -> bool:
    """
    Whether `var` occurs in a position that writes it: the target of an assignment (including a
    multi-assignment slot), the operand of a `++`/`--` update, the loop variable of a `foreach`, a
    parameter declaration, the operand of a `[ref]` cast, or a position an assignment stores
    *through*. Every other occurrence reads the variable.

    A store through is one of these although it installs nothing. `$x[0] = 'z'` leaves the name
    bound to the object it was bound to, but a read below it observes a different value, and that
    is the whole of what a write is to the layer that orders reads against writes. Counting it a
    read instead is what forced `Ps1VariableFlow` to give up on every occurrence of the name.
    """
    return occurrence_role(var).stores

def model_cache(transformer, root)

The pipeline's shared Ps1ModelCache for root when one is attached to transformer and built over that same root, otherwise a fresh cache stashed back onto transformer for reuse within its single-pass lifetime. See ModelCacheBase.for_transformer().

Expand source code Browse git

def model_cache(transformer: Transformer, root: Node) -> Ps1ModelCache:
    """
    The pipeline's shared `Ps1ModelCache` for *root* when one is attached to *transformer* and
    built over that same root, otherwise a fresh cache stashed back onto *transformer* for reuse
    within its single-pass lifetime. See
    `refinery.lib.scripts.modelcache.ModelCacheBase.for_transformer`.
    """
    return Ps1ModelCache.for_transformer(transformer, root)

def occurrence_role(var)

The Ps1OccurrenceRole of var.

The order the cases are tried in is the order they nest. An occurrence an assignment stores through is a target position as much as a plain target is, and is decided first because assignment_of deliberately answers None for it; a reference cast is decided last among the writes because everything above it is a syntactic position and a cast is a value form.

Expand source code Browse git

def occurrence_role(var: Ps1Variable) -> Ps1OccurrenceRole:
    """
    The `Ps1OccurrenceRole` of `var`.

    The order the cases are tried in is the order they nest. An occurrence an assignment stores
    through is a target position as much as a plain target is, and is decided first because
    `assignment_of` deliberately answers `None` for it; a reference cast is decided last among the
    writes because everything above it is a syntactic position and a cast is a value form.
    """
    if _is_member_declaration(var):
        return Ps1OccurrenceRole.NOT_A_REFERENCE
    if _stores_through(var) or _stores_through_a_call_slot(_enclosing_call_slot(var)):
        return Ps1OccurrenceRole.WRITE_THROUGH
    assignment = assignment_of(var)
    if assignment is not None:
        if assignment.operator == '=':
            return Ps1OccurrenceRole.WRITE_REPLACING
        return Ps1OccurrenceRole.WRITE_OBSERVING
    parent = var.parent
    if isinstance(parent, Ps1UnaryExpression) and parent.operator in ('++', '--'):
        if parent.operand is var:
            return Ps1OccurrenceRole.WRITE_OBSERVING
    if isinstance(parent, Ps1ForEachLoop) and parent.variable is var:
        return Ps1OccurrenceRole.WRITE_REPLACING
    if isinstance(parent, Ps1ParameterDeclaration) and parent.variable is var:
        return Ps1OccurrenceRole.WRITE_REPLACING
    if is_reference_cast(parent) and parent.operand is var:
        return Ps1OccurrenceRole.WRITE_OBSERVING
    return Ps1OccurrenceRole.READ

def replaces_value(var)

Whether var occupies the target position of a plain = assignment, which overwrites the variable without observing its previous value. The target of a compound assignment (+=, -=, .=, …) is excluded: it reads the variable as well as writing it.

Expand source code Browse git

def replaces_value(var: Ps1Variable) -> bool:
    """
    Whether `var` occupies the target position of a plain `=` assignment, which overwrites the
    variable without observing its previous value. The target of a compound assignment (`+=`, `-=`,
    `.=`, …) is excluded: it reads the variable as well as writing it.
    """
    assignment = assignment_of(var)
    return assignment is not None and assignment.operator == '='

def statement_effect(stmt, world)

Classify the observable effect of a standalone statement as a StatementEffect. This is the one shared authority the dead-code and junk-removal passes consult so they never disagree about whether a statement carries a body's output: a DISCARD emits nothing, an OUTPUT yields a value that emit-safety must protect in a captured body, and an EFFECT must always be kept.

DISCARD is a claim about emission and about nothing else. It used to be read as always safe to drop, which held only because the one thing it admitted was a discard idiom over a pure expression. A call returning System.Void emits as little and can still throw, so what decides whether dropping it changes anything is the removal veto — fault_is_observed — and not this.

Expand source code Browse git

def statement_effect(stmt, world: Ps1WorldReach) -> StatementEffect:
    """
    Classify the observable effect of a standalone statement as a `StatementEffect`. This is the one
    shared authority the dead-code and junk-removal passes consult so they never disagree about
    whether a statement carries a body's output: a `DISCARD` emits nothing, an `OUTPUT` yields a
    value that emit-safety must protect in a captured body, and an `EFFECT` must always be kept.

    **`DISCARD` is a claim about emission and about nothing else.** It used to be read as *always
    safe to drop*, which held only because the one thing it admitted was a discard idiom over a pure
    expression. A call returning `System.Void` emits as little and can still throw, so what decides
    whether dropping it changes anything is the removal veto — `fault_is_observed` — and not this.
    """
    if not isinstance(stmt, Ps1ExpressionStatement):
        return StatementEffect.EFFECT
    expr = stmt.expression
    if expr is None:
        return StatementEffect.DISCARD
    if _is_void_cast(expr):
        if is_side_effect_free(expr.operand, world):
            return StatementEffect.DISCARD
        return StatementEffect.EFFECT
    if isinstance(expr, Ps1Pipeline):
        # The prefix is walked exactly once and every branch below is derived from that one answer.
        # Asking `_pipeline_prefix_is_pure` per idiom and then falling through to
        # `is_side_effect_free(expr)` re-walks the same elements, and because a pipeline cmdlet
        # body re-enters here through `_command_body_is_pure`, that doubling compounds into 2^depth
        # work on the nested `... | ForEach-Object { ... } | Out-Null` shape.
        prefix_is_pure = _pipeline_prefix_is_pure(expr, world)
        if prefix_is_pure and (
            _pipeline_ends_with_out_null(expr, world)
            or _pipeline_ends_with_void_foreach(expr, world)
        ):
            return StatementEffect.DISCARD
        if _pipeline_ends_with_cmdlet(expr, _PURE_PIPELINE_CMDLETS):
            # A pure pipeline cmdlet (`... | Where-Object {...}`) yields a filtered value a caller
            # may consume, so it is kept even though it performs no side effect of its own.
            return StatementEffect.EFFECT
        if prefix_is_pure and _pipeline_final_is_pure(expr, world):
            return StatementEffect.OUTPUT
        return StatementEffect.EFFECT
    if _is_null_discard(expr):
        if expr.value is not None and is_side_effect_free(expr.value, world):
            return StatementEffect.DISCARD
        return StatementEffect.EFFECT
    if is_side_effect_free(expr, world):
        if _emits_nothing(expr, world):
            return StatementEffect.DISCARD
        return StatementEffect.OUTPUT
    return StatementEffect.EFFECT

Classes

class Binding (name, scope, reads=<factory>, writes=<factory>, dynamic_or_qualified=False, constraints=<factory>)

A single variable name bound within one scope. writes holds every occurrence that writes it (an assignment target, a ++/-- operand, a foreach variable, a parameter, a [ref], a command that addresses the name as a string, a store through the value such as the $x of $x[0] = 9 or of [Array]::Reverse($x), and a store shared in from another name for the same object); reads holds every occurrence that reads it, including a bare read that fell through from a nested block. dynamic_or_qualified marks a binding a scope qualifier or dynamic scope could reach with no occurrence in reads — conservatively kept live.

Not every occurrence in writes installs a value, so a consumer reading one has to ask. Occurrence.role says whether the store replaces the value or reaches through it, and Occurrence.shared_through whether the occurrence is spelled on this name at all.

Expand source code Browse git

@dataclass(eq=False)
class Binding:
    """
    A single variable name bound within one scope. `writes` holds every occurrence that writes it
    (an assignment target, a `++`/`--` operand, a `foreach` variable, a parameter, a `[ref]`, a
    command that addresses the name as a string, a store *through* the value such as the `$x` of
    `$x[0] = 9` or of `[Array]::Reverse($x)`, and a store shared in from another name for the same
    object); `reads` holds every occurrence that reads it, including a bare read that fell through
    from a nested block. `dynamic_or_qualified` marks a binding a scope qualifier or dynamic scope
    could reach with no occurrence in `reads` — conservatively kept live.

    Not every occurrence in `writes` installs a value, so a consumer reading one has to ask.
    `Occurrence.role` says whether the store replaces the value or reaches through it, and
    `Occurrence.shared_through` whether the occurrence is spelled on this name at all.
    """
    name: str
    scope: Scope
    reads: list[Occurrence] = field(default_factory=list)
    writes: list[Occurrence] = field(default_factory=list)
    dynamic_or_qualified: bool = False
    #: Every type a constrained write of this binding names — the `string` of `[string]$q = 5`.
    #: PowerShell stores the constraint on the *variable*, not on the write, so it converts what
    #: every later write stores as well: measured, `[string]$q = 5; $q = 1, 2, 3; $q.Length` is 5,
    #: because `$q` holds the String `1 2 3`, not the array. Empty for a name no write constrains.
    #:
    #: Resolved rather than spelled, so that `[string]` and `[System.String]` are the one constraint
    #: they are. A set of source spellings would read those two as a name constrained twice, which
    #: a consumer has to refuse outright — and spelling a type two ways is obfuscation rather than
    #: an oddity. A spelling the data resolves to nothing is `None`, which is a constraint whose
    #: conversion cannot be named and is refused on its own account.
    constraints: set[Ps1TypeName | None] = field(default_factory=set)

    @property
    def is_read(self) -> bool:
        """
        Whether any occurrence reads the binding's value.
        """
        return bool(self.reads)

    @property
    def uses(self) -> list[Occurrence]:
        """
        Every occurrence that observes the binding's value: its `reads`, and those of its `writes`
        that read what was there in order to write it.

        The two lists are buckets, not roles, and an occurrence that both reads and writes has no
        bucket of its own — `$x += 1` and `[ref]$x` are filed under `writes` and observe the value
        as surely as anything in `reads`. Every consumer deciding whether a value is still wanted
        asks this rather than `reads`, because asking `reads` is exactly how a store whose only
        reader is a compound assignment came to be deletable.
        """
        return [
            *self.reads,
            *(write for write in self.writes if write.role.observes),
        ]

    @property
    def is_dead(self) -> bool:
        """
        Whether no use observes the binding's value: no occurrence observes it and no qualifier or
        dynamic scope reaches it. The write occurrences of a dead binding can be removed when they
        carry no other side effect (which the caller decides).
        """
        return not self.uses and not self.dynamic_or_qualified

Instance variables

var name

The type of the None singleton.

var scope

The type of the None singleton.

var reads

The type of the None singleton.

var writes

The type of the None singleton.

var constraints

Resolved rather than spelled, so that [string] and [System.String] are the one constraint they are. A set of source spellings would read those two as a name constrained twice, which a consumer has to refuse outright — and spelling a type two ways is obfuscation rather than an oddity. A spelling the data resolves to nothing is None, which is a constraint whose conversion cannot be named and is refused on its own account.

var dynamic_or_qualified

The type of the None singleton.

var is_read

Whether any occurrence reads the binding's value.

Expand source code Browse git

@property
def is_read(self) -> bool:
    """
    Whether any occurrence reads the binding's value.
    """
    return bool(self.reads)

var uses

Every occurrence that observes the binding's value: its reads, and those of its writes that read what was there in order to write it.

The two lists are buckets, not roles, and an occurrence that both reads and writes has no bucket of its own — $x += 1 and [ref]$x are filed under writes and observe the value as surely as anything in reads. Every consumer deciding whether a value is still wanted asks this rather than reads, because asking reads is exactly how a store whose only reader is a compound assignment came to be deletable.

Expand source code Browse git

@property
def uses(self) -> list[Occurrence]:
    """
    Every occurrence that observes the binding's value: its `reads`, and those of its `writes`
    that read what was there in order to write it.

    The two lists are buckets, not roles, and an occurrence that both reads and writes has no
    bucket of its own — `$x += 1` and `[ref]$x` are filed under `writes` and observe the value
    as surely as anything in `reads`. Every consumer deciding whether a value is still wanted
    asks this rather than `reads`, because asking `reads` is exactly how a store whose only
    reader is a compound assignment came to be deletable.
    """
    return [
        *self.reads,
        *(write for write in self.writes if write.role.observes),
    ]

var is_dead

Whether no use observes the binding's value: no occurrence observes it and no qualifier or dynamic scope reaches it. The write occurrences of a dead binding can be removed when they carry no other side effect (which the caller decides).

Expand source code Browse git

@property
def is_dead(self) -> bool:
    """
    Whether no use observes the binding's value: no occurrence observes it and no qualifier or
    dynamic scope reaches it. The write occurrences of a dead binding can be removed when they
    carry no other side effect (which the caller decides).
    """
    return not self.uses and not self.dynamic_or_qualified

class Ps1ModelCache (root, options=None)

Lazily builds and memoizes the analysis models for one root script — the Ps1SemanticModel, the Ps1TypeWorld, the Ps1CallGraph, the Ps1OutputFlow, the ControlFlowModel, the Ps1FaultReach, the Ps1BlockModel and the CycleModel derived from it — each dropped whenever this root's AST-mutation counter advances past the value it was built at.

Expand source code Browse git

class Ps1ModelCache(ModelCacheBase):
    """
    Lazily builds and memoizes the analysis models for one root script — the
    `refinery.lib.scripts.ps1.analysis.model.Ps1SemanticModel`, the
    `refinery.lib.scripts.ps1.analysis.world.Ps1TypeWorld`, the
    `refinery.lib.scripts.ps1.analysis.callgraph.Ps1CallGraph`, the
    `refinery.lib.scripts.ps1.analysis.effects.Ps1OutputFlow`, the
    `refinery.lib.scripts.analysis.cfg.ControlFlowModel`, the
    `refinery.lib.scripts.ps1.analysis.faults.Ps1FaultReach`, the
    `refinery.lib.scripts.ps1.analysis.blocks.Ps1BlockModel` and the
    `refinery.lib.scripts.analysis.cycles.CycleModel` derived from it — each dropped
    whenever this root's AST-mutation counter advances past the value it was built at.
    """

    _SLOTS = (
        '_model',
        '_world_measurement',
        '_world_reach',
        '_call_graph',
        '_output_flow',
        '_control_flow',
        '_faults',
        '_dominance',
        '_blocks',
        '_cycles',
        '_variable_flow',
        '_commands',
    )

    root: Ps1Script
    _model: Ps1SemanticModel | None
    _world_measurement: Ps1WorldMeasurement | None
    _world_reach: Ps1WorldReach | None
    _call_graph: Ps1CallGraph | None
    _output_flow: Ps1OutputFlow | None
    _control_flow: ControlFlowModel | None
    _faults: Ps1FaultReach | None
    _dominance: DominatorModel | None
    _blocks: Ps1BlockModel | None
    _cycles: CycleModel | None
    _variable_flow: Ps1VariableFlow | None
    _commands: Ps1CommandModel | None

    @property
    def model(self) -> Ps1SemanticModel:
        return self._lazy('_model', lambda: build_semantic_model(self.root))

    @property
    def world_measurement(self) -> Ps1WorldMeasurement:
        """
        The one walk that reads whether this script leaves the .NET type system and the command
        table intact, which command names it takes over, and where every world-opening statement
        sits. `closed_world` projects the verdict from it and `world_reach` floods from its openers,
        so the whole-run answer and the positional one are never two different readings of the tree.

        The run's `refinery.lib.scripts.ps1.options.Ps1DeobfuscationOptions` are read from the cache
        because the walk is performed once per cache. A cache made without them — a model built
        standalone in a test — measures the suspecting model, which is the sound answer.
        """
        return self._lazy('_world_measurement', lambda: measure_world(self.root, self.options))

    @property
    def closed_world(self) -> Ps1TypeWorld:
        """
        Whether this script leaves the .NET type system and the command table intact, and which
        command names it takes over. `refinery.lib.scripts.ps1.analysis.effects` takes this as the
        context of every purity verdict, and every verdict in a run must be asked against the same
        one, or two transforms reach opposite conclusions about the same node.
        """
        return self.world_measurement.world

    @property
    def world_reach(self) -> Ps1WorldReach:
        """
        The flow-sensitive reading of `closed_world`: whether the type world is closed, and a
        command name still trustworthy, at one particular node, over `control_flow`. The effect
        layer takes this in place of the leaf world so a member-read grant or a pure-command
        verdict may survive a leak the node provably runs before. Rebuilt with the rest of the
        cache when this root's tree changes, so a transform never reads a position against a stale
        graph. The control-flow model is passed as a thunk so a script that neither opens the
        world nor redefines a command — one with nothing to flood from — never pays to build a
        graph its reach model would not read.
        """
        return self._lazy('_world_reach', lambda: build_world_reach(
            self.world_measurement, lambda: self.control_flow))

    @property
    def call_graph(self) -> Ps1CallGraph:
        """
        Which definitions a command name reaches and which invocations reach it, over this root. The
        `closed_world` is a parameter of the build rather than of the queries because the verdict it
        supplies is one of the reasons the graph declares itself unreadable, and a graph that
        answered that differently per caller would be two graphs.
        """
        return self._lazy(
            '_call_graph',
            lambda: build_call_graph(self.root, self.closed_world, self.options),
        )

    @property
    def output_flow(self) -> Ps1OutputFlow:
        """
        Where each function body's output ends up, joined over the call sites in `call_graph`.
        Every pass that deletes a write to the output stream reads this one, so no two of them can
        disagree about who was going to see the value.
        """
        return self._lazy('_output_flow', lambda: build_output_flow(self.call_graph))

    @property
    def control_flow(self) -> ControlFlowModel:
        """
        One control-flow graph per script block and one for the script itself, over this root — see
        `refinery.lib.scripts.ps1.analysis.cfg.FUNCTION_NODES` for what owns one and why.

        Purely syntactic, so it needs none of the models above and nothing about the order they are
        built in matters. What it answers is the question every pass here has been approximating
        privately: whether one statement runs before another, whether a branch runs at all, and
        whether a handler is still reachable once a body is emptied.
        """
        return self._lazy('_control_flow', lambda: build_control_flow_model(self.root))

    @property
    def faults(self) -> Ps1FaultReach:
        """
        Where a terminating error raised at a point in this root goes, over `control_flow`. The
        single place a pass asks whether deleting something changes which handler runs — the
        question every removing pass used to answer for itself by looking at the statement's
        immediate holder, which reads a handler one nesting level away as no handler at all.

        Purely syntactic like the graphs it reads, so it joins nothing else and orders nothing else.
        """
        return self._lazy('_faults', lambda: build_fault_reach(self.control_flow))

    @property
    def dominance(self) -> DominatorModel:
        """
        Whether one statement of this root is guaranteed to have executed by the time another runs,
        over `control_flow`. The single place that ordering is answered: a pass that needs to know a
        write runs before a read, or that a statement after a `return` cannot be reached, asks here
        rather than reconstructing the relation from the tree.
        """
        return self._lazy('_dominance', lambda: build_dominance(self.control_flow))

    @property
    def blocks(self) -> Ps1BlockModel:
        """
        Where each script block of this root runs — at what point, in whose scope, how many times.
        Reads the whole-run shadow set from `closed_world` so a body handed to a `ForEach-Object` or
        `Where-Object` the script has redefined is placed as data, and is otherwise syntactic like
        `control_flow` — the answer three other layers used to guess from the code a block is
        *written* in.
        """
        return self._lazy('_blocks', lambda: build_block_model(
            self.root, self.closed_world.shadowed_names))

    @property
    def cycles(self) -> CycleModel:
        """
        Which points of this script can be reached more than once, over `control_flow`. A pass that
        establishes a fact from one visit to a statement — a variable's value, a stream's contents —
        asks this before carrying it to a reader, because a point control returns to has no single
        value to carry.

        It is built over `blocks` so that a body run by a cmdlet that enumerates is known to repeat.
        Without that the walk out of a block follows it to where its value was written, and a
        `ForEach-Object` body reads as running exactly once.
        """
        return self._lazy('_cycles', lambda: CycleModel(self.control_flow, self.blocks.body_site))

    @property
    def commands(self) -> Ps1CommandModel:
        """
        What each command invocation of this root denotes — a name, nothing, or unknown — over
        `control_flow` and `dominance`. The one place command identity is answered: a pass that
        rewrites an alias, folds a call to a function, or reads which command a name runs asks here
        rather than splitting the alias relation or ignoring the precedence that makes a default
        alias beat a script function. It reads the script's function names from `call_graph`, the
        layer that already owns which definitions a name denotes, and the wider set of names the
        script takes over from `closed_world`, so the two models agree on what has been shadowed.
        """
        return self._lazy('_commands', lambda: build_command_model(
            self.root, self.control_flow, self.dominance, self.blocks,
            frozenset(self.call_graph.defined_names), self.closed_world.shadowed_names))

    @property
    def variable_flow(self) -> Ps1VariableFlow:
        """
        Which write each variable read observes, over `model`, `control_flow`, `dominance`, `blocks`
        and `cycles`. The one place that question is answered: a pass that decides what a name holds
        at a point asks here rather than walking the tree for an assignment that looks near enough.
        """
        return self._lazy('_variable_flow', lambda: build_variable_flow(
            self.model, self.control_flow, self.dominance, self.blocks, self.cycles))

Ancestors

• ModelCacheBase

Instance variables

var model

Expand source code Browse git

@property
def model(self) -> Ps1SemanticModel:
    return self._lazy('_model', lambda: build_semantic_model(self.root))

var world_measurement

The one walk that reads whether this script leaves the .NET type system and the command table intact, which command names it takes over, and where every world-opening statement sits. closed_world projects the verdict from it and world_reach floods from its openers, so the whole-run answer and the positional one are never two different readings of the tree.

The run's Ps1DeobfuscationOptions are read from the cache because the walk is performed once per cache. A cache made without them — a model built standalone in a test — measures the suspecting model, which is the sound answer.

Expand source code Browse git

@property
def world_measurement(self) -> Ps1WorldMeasurement:
    """
    The one walk that reads whether this script leaves the .NET type system and the command
    table intact, which command names it takes over, and where every world-opening statement
    sits. `closed_world` projects the verdict from it and `world_reach` floods from its openers,
    so the whole-run answer and the positional one are never two different readings of the tree.

    The run's `refinery.lib.scripts.ps1.options.Ps1DeobfuscationOptions` are read from the cache
    because the walk is performed once per cache. A cache made without them — a model built
    standalone in a test — measures the suspecting model, which is the sound answer.
    """
    return self._lazy('_world_measurement', lambda: measure_world(self.root, self.options))

var closed_world

Whether this script leaves the .NET type system and the command table intact, and which command names it takes over. refinery.lib.scripts.ps1.analysis.effects takes this as the context of every purity verdict, and every verdict in a run must be asked against the same one, or two transforms reach opposite conclusions about the same node.

Expand source code Browse git

@property
def closed_world(self) -> Ps1TypeWorld:
    """
    Whether this script leaves the .NET type system and the command table intact, and which
    command names it takes over. `refinery.lib.scripts.ps1.analysis.effects` takes this as the
    context of every purity verdict, and every verdict in a run must be asked against the same
    one, or two transforms reach opposite conclusions about the same node.
    """
    return self.world_measurement.world

var world_reach

The flow-sensitive reading of closed_world: whether the type world is closed, and a command name still trustworthy, at one particular node, over control_flow. The effect layer takes this in place of the leaf world so a member-read grant or a pure-command verdict may survive a leak the node provably runs before. Rebuilt with the rest of the cache when this root's tree changes, so a transform never reads a position against a stale graph. The control-flow model is passed as a thunk so a script that neither opens the world nor redefines a command — one with nothing to flood from — never pays to build a graph its reach model would not read.

Expand source code Browse git

@property
def world_reach(self) -> Ps1WorldReach:
    """
    The flow-sensitive reading of `closed_world`: whether the type world is closed, and a
    command name still trustworthy, at one particular node, over `control_flow`. The effect
    layer takes this in place of the leaf world so a member-read grant or a pure-command
    verdict may survive a leak the node provably runs before. Rebuilt with the rest of the
    cache when this root's tree changes, so a transform never reads a position against a stale
    graph. The control-flow model is passed as a thunk so a script that neither opens the
    world nor redefines a command — one with nothing to flood from — never pays to build a
    graph its reach model would not read.
    """
    return self._lazy('_world_reach', lambda: build_world_reach(
        self.world_measurement, lambda: self.control_flow))

var call_graph

Which definitions a command name reaches and which invocations reach it, over this root. The closed_world is a parameter of the build rather than of the queries because the verdict it supplies is one of the reasons the graph declares itself unreadable, and a graph that answered that differently per caller would be two graphs.

Expand source code Browse git

@property
def call_graph(self) -> Ps1CallGraph:
    """
    Which definitions a command name reaches and which invocations reach it, over this root. The
    `closed_world` is a parameter of the build rather than of the queries because the verdict it
    supplies is one of the reasons the graph declares itself unreadable, and a graph that
    answered that differently per caller would be two graphs.
    """
    return self._lazy(
        '_call_graph',
        lambda: build_call_graph(self.root, self.closed_world, self.options),
    )

var output_flow

Where each function body's output ends up, joined over the call sites in call_graph. Every pass that deletes a write to the output stream reads this one, so no two of them can disagree about who was going to see the value.

Expand source code Browse git

@property
def output_flow(self) -> Ps1OutputFlow:
    """
    Where each function body's output ends up, joined over the call sites in `call_graph`.
    Every pass that deletes a write to the output stream reads this one, so no two of them can
    disagree about who was going to see the value.
    """
    return self._lazy('_output_flow', lambda: build_output_flow(self.call_graph))

var control_flow

One control-flow graph per script block and one for the script itself, over this root — see FUNCTION_NODES for what owns one and why.

Purely syntactic, so it needs none of the models above and nothing about the order they are built in matters. What it answers is the question every pass here has been approximating privately: whether one statement runs before another, whether a branch runs at all, and whether a handler is still reachable once a body is emptied.

Expand source code Browse git

@property
def control_flow(self) -> ControlFlowModel:
    """
    One control-flow graph per script block and one for the script itself, over this root — see
    `refinery.lib.scripts.ps1.analysis.cfg.FUNCTION_NODES` for what owns one and why.

    Purely syntactic, so it needs none of the models above and nothing about the order they are
    built in matters. What it answers is the question every pass here has been approximating
    privately: whether one statement runs before another, whether a branch runs at all, and
    whether a handler is still reachable once a body is emptied.
    """
    return self._lazy('_control_flow', lambda: build_control_flow_model(self.root))

var faults

Where a terminating error raised at a point in this root goes, over control_flow. The single place a pass asks whether deleting something changes which handler runs — the question every removing pass used to answer for itself by looking at the statement's immediate holder, which reads a handler one nesting level away as no handler at all.

Purely syntactic like the graphs it reads, so it joins nothing else and orders nothing else.

Expand source code Browse git

@property
def faults(self) -> Ps1FaultReach:
    """
    Where a terminating error raised at a point in this root goes, over `control_flow`. The
    single place a pass asks whether deleting something changes which handler runs — the
    question every removing pass used to answer for itself by looking at the statement's
    immediate holder, which reads a handler one nesting level away as no handler at all.

    Purely syntactic like the graphs it reads, so it joins nothing else and orders nothing else.
    """
    return self._lazy('_faults', lambda: build_fault_reach(self.control_flow))

var dominance

Whether one statement of this root is guaranteed to have executed by the time another runs, over control_flow. The single place that ordering is answered: a pass that needs to know a write runs before a read, or that a statement after a return cannot be reached, asks here rather than reconstructing the relation from the tree.

Expand source code Browse git

@property
def dominance(self) -> DominatorModel:
    """
    Whether one statement of this root is guaranteed to have executed by the time another runs,
    over `control_flow`. The single place that ordering is answered: a pass that needs to know a
    write runs before a read, or that a statement after a `return` cannot be reached, asks here
    rather than reconstructing the relation from the tree.
    """
    return self._lazy('_dominance', lambda: build_dominance(self.control_flow))

var blocks

Where each script block of this root runs — at what point, in whose scope, how many times. Reads the whole-run shadow set from closed_world so a body handed to a ForEach-Object or Where-Object the script has redefined is placed as data, and is otherwise syntactic like control_flow — the answer three other layers used to guess from the code a block is written in.

Expand source code Browse git

@property
def blocks(self) -> Ps1BlockModel:
    """
    Where each script block of this root runs — at what point, in whose scope, how many times.
    Reads the whole-run shadow set from `closed_world` so a body handed to a `ForEach-Object` or
    `Where-Object` the script has redefined is placed as data, and is otherwise syntactic like
    `control_flow` — the answer three other layers used to guess from the code a block is
    *written* in.
    """
    return self._lazy('_blocks', lambda: build_block_model(
        self.root, self.closed_world.shadowed_names))

var cycles

Which points of this script can be reached more than once, over control_flow. A pass that establishes a fact from one visit to a statement — a variable's value, a stream's contents — asks this before carrying it to a reader, because a point control returns to has no single value to carry.

It is built over refinery.lib.scripts.ps1.analysis.blocks so that a body run by a cmdlet that enumerates is known to repeat. Without that the walk out of a block follows it to where its value was written, and a ForEach-Object body reads as running exactly once.

Expand source code Browse git

@property
def cycles(self) -> CycleModel:
    """
    Which points of this script can be reached more than once, over `control_flow`. A pass that
    establishes a fact from one visit to a statement — a variable's value, a stream's contents —
    asks this before carrying it to a reader, because a point control returns to has no single
    value to carry.

    It is built over `blocks` so that a body run by a cmdlet that enumerates is known to repeat.
    Without that the walk out of a block follows it to where its value was written, and a
    `ForEach-Object` body reads as running exactly once.
    """
    return self._lazy('_cycles', lambda: CycleModel(self.control_flow, self.blocks.body_site))

var commands

What each command invocation of this root denotes — a name, nothing, or unknown — over control_flow and refinery.lib.scripts.ps1.analysis.dominance. The one place command identity is answered: a pass that rewrites an alias, folds a call to a function, or reads which command a name runs asks here rather than splitting the alias relation or ignoring the precedence that makes a default alias beat a script function. It reads the script's function names from call_graph, the layer that already owns which definitions a name denotes, and the wider set of names the script takes over from closed_world, so the two models agree on what has been shadowed.

Expand source code Browse git

@property
def commands(self) -> Ps1CommandModel:
    """
    What each command invocation of this root denotes — a name, nothing, or unknown — over
    `control_flow` and `dominance`. The one place command identity is answered: a pass that
    rewrites an alias, folds a call to a function, or reads which command a name runs asks here
    rather than splitting the alias relation or ignoring the precedence that makes a default
    alias beat a script function. It reads the script's function names from `call_graph`, the
    layer that already owns which definitions a name denotes, and the wider set of names the
    script takes over from `closed_world`, so the two models agree on what has been shadowed.
    """
    return self._lazy('_commands', lambda: build_command_model(
        self.root, self.control_flow, self.dominance, self.blocks,
        frozenset(self.call_graph.defined_names), self.closed_world.shadowed_names))

var variable_flow

Which write each variable read observes, over refinery.lib.scripts.ps1.analysis.model, control_flow, refinery.lib.scripts.ps1.analysis.dominance, refinery.lib.scripts.ps1.analysis.blocks and cycles. The one place that question is answered: a pass that decides what a name holds at a point asks here rather than walking the tree for an assignment that looks near enough.

Expand source code Browse git

@property
def variable_flow(self) -> Ps1VariableFlow:
    """
    Which write each variable read observes, over `model`, `control_flow`, `dominance`, `blocks`
    and `cycles`. The one place that question is answered: a pass that decides what a name holds
    at a point asks here rather than walking the tree for an assignment that looks near enough.
    """
    return self._lazy('_variable_flow', lambda: build_variable_flow(
        self.model, self.control_flow, self.dominance, self.blocks, self.cycles))

Inherited members

• ModelCacheBase:
  • for_transformer
  • options
  • pinned
  • root

class Ps1OccurrenceRole (stores, observes, substitutable, through)

What one occurrence of a variable does to the value the name holds. Every occurrence has exactly one role, and the transforms ask for it rather than each assembling an answer from a handful of positional predicates — which is how [ref]$n came to be read as a plain read by every one of them at once.

NOT_A_REFERENCE — the occurrence does not reference a variable at all: a class property member declaration names a member of the class, a namespace of its own. READ — observes the value and does not change it. WRITE_REPLACING — stores without observing what was there: $x = v, a foreach variable, a parameter. WRITE_OBSERVING — stores and observes: $x += v, $x++, and [ref]$x, whose callee may store back through the wrapper it is handed. WRITE_THROUGH — reads the variable to reach a place inside it that is written: the $x of $x[0] = 'z' or $x.Length = 5. The name still holds whatever it held, so this observes the value like a read and installs none of its own; but what it holds is no longer what it held, so it is a write of the binding all the same and no value may be installed in its place.

Each member carries the four answers a consumer needs, in the order the fields are declared below, rather than being compared against a list of members at every site. A question answered by a membership test lives at each of its call sites and has to be found again by grep whenever a role is added or its meaning moves; a field lives here, beside the member it is about.

enum.unique because the answers are the value: two members answering alike would be one member under two names, dispatching as whichever was declared first while every is test against the other still passed.

Expand source code Browse git

@enum.unique
class Ps1OccurrenceRole(enum.Enum):
    """
    What one occurrence of a variable does to the value the name holds. Every occurrence has exactly
    one role, and the transforms ask for it rather than each assembling an answer from a handful of
    positional predicates — which is how `[ref]$n` came to be read as a plain read by every one of
    them at once.

    `NOT_A_REFERENCE` — the occurrence does not reference a variable at all: a class property member
    declaration names a member of the class, a namespace of its own.
    `READ` — observes the value and does not change it.
    `WRITE_REPLACING` — stores without observing what was there: `$x = v`, a `foreach` variable, a
    parameter.
    `WRITE_OBSERVING` — stores *and* observes: `$x += v`, `$x++`, and `[ref]$x`, whose callee may
    store back through the wrapper it is handed.
    `WRITE_THROUGH` — reads the variable to reach a place inside it that is written: the `$x` of
    `$x[0] = 'z'` or `$x.Length = 5`. The name still holds whatever it held, so this observes the
    value like a read and installs none of its own; but *what* it holds is no longer what it held,
    so it is a write of the binding all the same and no value may be installed in its place.

    Each member carries the four answers a consumer needs, in the order the fields are declared
    below, rather than being compared against a list of members at every site. A question answered
    by a membership test lives at each of its call sites and has to be found again by grep whenever
    a role is added or its meaning moves; a field lives here, beside the member it is about.

    `enum.unique` because the answers *are* the value: two members answering alike would be one
    member under two names, dispatching as whichever was declared first while every `is` test
    against the other still passed.
    """
    NOT_A_REFERENCE = (False, False, False, False)
    READ            = (False, True, True, False)    # noqa
    WRITE_REPLACING = (True, False, False, False)
    WRITE_OBSERVING = (True, True, False, False)
    WRITE_THROUGH   = (True, True, False, True)     # noqa

    #: Whether the occurrence is filed among the binding's writes, because it changes what a read
    #: below it observes.
    stores: bool
    #: Whether the occurrence observes the value the name holds.
    observes: bool
    #: Whether a value may be installed in the occurrence's place. Only a plain read, and even then
    #: `is_substitutable_position` has a caveat of its own to add.
    substitutable: bool
    #: Whether the occurrence reaches a place *inside* the value rather than the binding itself, so
    #: that the name is left holding whatever it held.
    through: bool

    def __init__(self, stores: bool, observes: bool, substitutable: bool, through: bool):
        self.stores = stores
        self.observes = observes
        self.substitutable = substitutable
        self.through = through

Ancestors

• enum.Enum

Class variables

var stores

Whether the occurrence is filed among the binding's writes, because it changes what a read below it observes.

var observes

Whether the occurrence observes the value the name holds.

var substitutable

Whether a value may be installed in the occurrence's place. Only a plain read, and even then is_substitutable_position() has a caveat of its own to add.

var through

Whether the occurrence reaches a place inside the value rather than the binding itself, so that the name is left holding whatever it held.

var NOT_A_REFERENCE

The type of the None singleton.

var READ

The type of the None singleton.

var WRITE_REPLACING

The type of the None singleton.

var WRITE_OBSERVING

The type of the None singleton.

var WRITE_THROUGH

The type of the None singleton.

class Ps1SemanticModel (root)

The resolved scope/binding/def-use model for one PowerShell script. Build it with build_semantic_model() and query it through scope_of and binding_of, through the bindings of a Scope, and — for the flow-sensitive dead-store sweep — through reads_in_scope and variables_in_scope.

Expand source code Browse git

class Ps1SemanticModel:
    """
    The resolved scope/binding/def-use model for one PowerShell script. Build it with
    `build_semantic_model` and query it through `scope_of` and `binding_of`, through the `bindings`
    of a `Scope`, and — for the flow-sensitive dead-store sweep — through `reads_in_scope` and
    `variables_in_scope`.
    """

    def __init__(self, root: Ps1Script):
        self.root = root
        self._node_scope: dict[int, Scope] = {}
        self._binding_of: dict[int, Binding] = {}
        self.root_scope = Scope(kind=ScopeKind.SCRIPT, node=root)
        self._node_scope[id(root)] = self.root_scope
        self._changes_in_place: bool | None = None
        self._populate(self.root_scope)
        self._build_def_use()

    @property
    def script_scope(self) -> Scope:
        """
        The scope the script itself introduces — the outermost scope, whose bindings are the
        script-level variables.
        """
        return self.root_scope

    @property
    def changes_an_object_in_place(self) -> bool:
        """
        Whether any occurrence in the script stores *through* a value rather than into a name — the
        `$x` of `$x[0] = 9`, of `$h.k = 9` and of `[Array]::Reverse($x)`.

        A script with none never changes an object after it is built, so there a copy of one and a
        second name for it are indistinguishable, and a consumer weighing the two may stop asking.
        That is the question every sharing guard is really about, and it is a fact about the script
        rather than about the name a guard happens to be standing on: an object handed to a
        hashtable key, to a property or to a callee is changed under a name the guard cannot see.
        """
        if self._changes_in_place is None:
            self._changes_in_place = any(
                write.role.through for write in self._every_write())
        return self._changes_in_place

    def scope_of(self, node: Node) -> Scope | None:
        """
        The innermost scope that contains *node*, or `None` if the node was not part of the script
        the model was built from. A node in an `if`/loop/`try` body resolves to the enclosing script
        or scriptblock scope, since those bodies introduce no scope of their own.
        """
        return self._node_scope.get(id(node))

    def binding_of(self, var: Ps1Variable) -> Binding | None:
        """
        The binding a variable occurrence resolves to — for a write, the binding in its defining
        scope; for a bare read, the nearest enclosing binding of the name — or `None` when the
        occurrence is free (an automatic or external variable the model never binds) or names a
        namespace outside the script's variables.
        """
        return self._binding_of.get(id(var))

    def reads_in_scope(self, node: Node, scope: Scope) -> set[str]:
        """
        The names of *scope*'s bindings read anywhere within *node*'s subtree — every bare read of a
        name *scope* binds, including one nested in a scriptblock, but not the target of a plain `=`
        assignment, which replaces the value without observing it. A compound-assignment target
        (`$x += 1`) does observe it and counts as a read. This is the read set the dead-store sweep
        flushes pending stores against: unlike the walk it replaces, it does not stop at a nested
        scriptblock, so a store read only through a captured block is correctly seen as live.
        """
        names: set[str] = set()
        for descendant in node.walk():
            if not isinstance(descendant, Ps1Variable):
                continue
            if descendant.scope is not Ps1ScopeModifier.NONE:
                continue
            name = descendant.name.lower()
            if name in scope.bindings and not replaces_value(descendant):
                names.add(name)
        return names

    def variables_in_scope(self, node: Node, scope: Scope) -> set[str]:
        """
        The names of *scope*'s bindings referenced in any way — read or written — within *node*'s
        subtree. The conservative flush set for a control-flow statement whose internal effect on a
        variable the linear sweep does not model: any mention of a bound name defers its pending
        store.
        """
        names: set[str] = set()
        for descendant in node.walk():
            if isinstance(descendant, Ps1Variable) and descendant.scope is Ps1ScopeModifier.NONE:
                name = descendant.name.lower()
                if name in scope.bindings:
                    names.add(name)
        return names

    def _populate(self, scope: Scope):
        for node in scope_local_nodes(scope.node):
            if isinstance(node, Ps1ScriptBlock):
                child = Scope(kind=self._scriptblock_kind(node), node=node, parent=scope)
                scope.children.append(child)
                self._node_scope[id(node)] = child
                self._populate(child)
                continue
            self._node_scope[id(node)] = scope
            if isinstance(node, Ps1Variable) and declares_binding(node):
                self._declare(node, scope)
            elif isinstance(node, Ps1CommandInvocation):
                self._declare_named(node, scope)

    @staticmethod
    def _scriptblock_kind(node: Ps1ScriptBlock) -> ScopeKind:
        if isinstance(node.parent, Ps1FunctionDefinition) and node.parent.body is node:
            return ScopeKind.FUNCTION
        return ScopeKind.SCRIPTBLOCK

    def _declare(self, var: Ps1Variable, current: Scope):
        scope = self._defining_scope(var, current)
        if scope is None:
            return
        key = binding_key(var)
        if key not in scope.bindings:
            scope.bindings[key] = Binding(name=key, scope=scope)

    def _declare_named(self, cmd: Ps1CommandInvocation, current: Scope):
        """
        Create the bindings a command addresses by string, and record a name it addresses that
        cannot be read.

        This is why the census is consulted while the model is built rather than applied to it
        afterwards: `Get-Process -OutVariable x` in a script that never writes `$x` any other way is
        the only mention of the name there is, so nothing exists to hang the reference on unless the
        binding is created here.

        An unreadable name landing in the command's own scope is *not* recorded here. That write
        happens at a point, and a point is what
        `refinery.lib.scripts.ps1.analysis.dataflow.Ps1VariableFlow.unattributable_writes` holds, so
        a read before it keeps the value it would have observed anyway. Only a write this cannot
        place against the reads it may reach — one aimed at the script scope, or at a scope the
        lexical chain cannot name — is a fact about the scope as a whole.
        """
        unreadable = unreadable_name_target(cmd)
        if unreadable is not None and unreadable is not Ps1NameTarget.LOCAL:
            self._doubt(unreadable, current)
        for reference in named_references(cmd):
            if reference.role is Ps1NameRole.READS:
                continue
            scope = self._named_scope(reference, current)
            if scope is None:
                continue
            if reference.key not in scope.bindings:
                scope.bindings[reference.key] = Binding(name=reference.key, scope=scope)

    def _named_scope(self, reference: Ps1NamedReference, current: Scope) -> Scope | None:
        """
        The scope a named reference resolves in: the one the command is written in for the measured
        default, the script scope for an explicitly script- or global-qualified form, and none at
        all for a target the lexical chain cannot name — `-Scope 1` writes the *caller's* scope,
        which is not an ancestor of anything here. An unplaceable write is recorded on the scope
        instead, where it puts every name in doubt rather than the wrong one.
        """
        if reference.target is Ps1NameTarget.SCRIPT:
            return self.root_scope
        if reference.target is Ps1NameTarget.LOCAL:
            return current
        self._doubt(Ps1NameTarget.UNREADABLE, current)
        return None

    def _doubt(self, target: Ps1NameTarget, current: Scope) -> None:
        """
        Record that a write nobody can attribute lands in *target*, so every binding it could reach
        is in doubt.

        A target the lexical chain cannot name reaches anywhere, and the script scope is the one
        scope every other can see through, so it is marked as well as the scope holding the command:
        under-marking here is a fold across a write, which is the direction that corrupts.
        """
        if target is Ps1NameTarget.SCRIPT:
            self.root_scope.writes_unreadable_names = True
            return
        current.writes_unreadable_names = True
        if target is Ps1NameTarget.UNREADABLE:
            self.root_scope.writes_unreadable_names = True

    def _defining_scope(self, var: Ps1Variable, current: Scope) -> Scope | None:
        """
        The scope a write to *var* binds. A bare, `$local:`, or `$private:` assignment binds in the
        current scope (write-local); a `$script:`, `$global:`, or `$using:` assignment, and an
        `$env:` assignment (a process-global environment variable, bound under an `env:`-prefixed
        key), bind at the script scope. The provider namespaces (`variable:`, `function:`,
        `alias:`, `drive:`) name a namespace distinct from script variables and bind nothing here.
        """
        modifier = var.scope
        if modifier in (Ps1ScopeModifier.NONE, Ps1ScopeModifier.LOCAL, Ps1ScopeModifier.PRIVATE):
            return current
        if modifier in (
            Ps1ScopeModifier.SCRIPT,
            Ps1ScopeModifier.GLOBAL,
            Ps1ScopeModifier.USING,
            Ps1ScopeModifier.ENV,
        ):
            return self.root_scope
        return None

    def _build_def_use(self):
        for node in self.root.walk():
            scope = self._node_scope.get(id(node))
            if scope is None:
                continue
            if isinstance(node, Ps1CommandInvocation):
                self._attribute_named(node, scope)
                continue
            if not isinstance(node, Ps1Variable) or _is_member_declaration(node):
                continue
            role = occurrence_role(node)
            if not role.stores:
                self._attribute_read(node, scope)
            elif role.through:
                self._attribute_write_through(node, scope)
            elif declares_binding(node):
                self._attribute_write(node, scope)
            else:
                self._attribute_reference(node, scope)
        self._record_type_constraints()
        self._share_stores_through_aliases()

    def _record_type_constraints(self):
        """
        File the type each constrained write names against the binding it writes.

        The constraint outlives the statement that carries it: `[string]$q = 5` stores an
        `ArgumentTypeConverterAttribute` on the variable, and every later write is converted through
        it. So this is a fact about the binding rather than about the occurrence, and a caller
        reading a value out of an *unconstrained* write of a constrained name has to know.
        """
        for binding in self._every_binding():
            for write in binding.writes:
                if write.may_define or not isinstance(write.node, Ps1Variable):
                    continue
                named = _constraint_on(write.node)
                if named is not None:
                    binding.constraints.add(data.resolve_type(named))

    def _every_binding(self) -> Iterator[Binding]:
        stack = [self.root_scope]
        while stack:
            scope = stack.pop()
            stack.extend(scope.children)
            yield from scope.bindings.values()

    def _share_stores_through_aliases(self):
        """
        File every store-through against each binding that names the same object.

        `$y = $x` does not copy the array; it gives the one array a second name. So
        `[Array]::Reverse($x)` changes what a read of `$y` observes and `$y[0] = 9` changes what a
        read of `$x` observes, and neither is an occurrence of the other name. This is Chow's χ —
        a may-def filed against every member of an alias class — at the one depth a syntactic model
        can see: a definition whose whole value is a bare variable.

        **A shared store-through is a may-def and nothing stronger, which is what keeps it from
        corrupting.** Whether the alias still holds where the store runs is an ordering question,
        and this layer sees no order: `$x = 1, 2, 3; $y = $x; $y = 9, 9, 9; [Array]::Reverse($x)`
        leaves `$y` holding `9, 9, 9` — measured — although the definition `$y = $x` is filed all
        the same. So every occurrence shared here carries the chain of definitions it came through,
        and only kills until
        `refinery.lib.scripts.ps1.analysis.dataflow.Ps1VariableFlow.reaching_definition` finds that
        chain intact at the store. Filing one where the alias does not hold then costs a fold and
        can never install a value the name never held.

        The class is closed under the relation rather than read off one definition, because a name
        reached through two of them is reached: `$y = $x; $z = $y` gives all three the one array,
        and a member that missed a store it did receive is the direction that answers with the value
        from before. The chain is what the second link is checked by: filed for `$z`, the store on
        `$x` carries both definitions, and either one broken breaks the answer.
        """
        for members, links in self._alias_classes():
            stores = [
                (binding, write)
                for binding in members
                for write in binding.writes
                if write.role.through and not write.may_define
            ]
            if not stores:
                continue
            adjacent = _link_adjacency(links)
            reached: dict[int, dict[int, tuple[Ps1AliasLink, ...]]] = {}
            for source, _ in stores:
                if id(source) not in reached:
                    reached[id(source)] = _alias_chains_from(adjacent, source)
            for binding in members:
                filed = {id(write.node) for write in binding.writes}
                for source, write in stores:
                    if source is binding or id(write.node) in filed:
                        continue
                    chain = reached[id(source)].get(id(binding))
                    if chain is None:
                        continue
                    filed.add(id(write.node))
                    binding.writes.append(
                        Occurrence(write.node, write.role, binding.name, chain))

    def _alias_classes(self) -> Iterator[tuple[list[Binding], list[Ps1AliasLink]]]:
        """
        The bindings a chain of definitions gives one object to, grouped, each beside the
        definitions that joined them. A definition qualifies when its whole value is a bare variable
        read — `$y = $x`, and `$y = ($x)`, since a parenthesis hands over what it wraps — or that
        read under a conversion, which qualifies as an *uncertain* link.

        One shape that reads like a definition is refused outright. A subexpression is not a
        parenthesis: measured, `$y = $($x)` unrolls the array to the pipeline and collects a fresh
        one, and `[object]::ReferenceEquals($x, $y)` is `False`. That is not doubt about whether the
        two names share, it is a definition that certainly copies, and filing it would cost folds
        for nothing.
        """
        classes: dict[int, list[Binding]] = {}
        joined: dict[int, list[Ps1AliasLink]] = {}
        for link in self._alias_definitions():
            here = classes.setdefault(id(link.first), [link.first])
            there = classes.get(id(link.second))
            if there is not here:
                if there is None:
                    here.append(link.second)
                else:
                    here.extend(there)
                    joined.setdefault(id(here), []).extend(joined.pop(id(there), []))
                for binding in there or (link.second,):
                    classes[id(binding)] = here
            joined.setdefault(id(here), []).append(link)
        seen: set[int] = set()
        for members in classes.values():
            if id(members) in seen:
                continue
            seen.add(id(members))
            yield members, joined.get(id(members), [])

    def _alias_definitions(self) -> Iterator[Ps1AliasLink]:
        """
        Each definition that hands one object to a second name, as the link between the two
        bindings it joins.

        A conversion on either side of the `=` is passed through and makes the link uncertain: it is
        the same object where nothing needed converting and a fresh one where something did, and
        which of the two ran is a question about the operand's runtime type. Both spellings count —
        `$y = [array]$x` and `$y = $x -as [array]` convert the value, `[int[]]$y = $x` constrains
        the variable — because the object either name ends up holding is the same question in all
        three. Measured: `$x = 1, 2, 3; $y = [array]$x; $y[0] = 9` leaves `$x` reading `9 2 3`.

        A constraint the *target* binding carries counts as much as one this occurrence spells,
        because PowerShell stores it on the variable and converts every later write through it.
        Measured: `[string]$y = 0; $x = 1, 2, 3; $y = $x; [Array]::Reverse($x); $y` writes `1 2 3`,
        because `$y` was handed the String `1 2 3` and never the array — so the definition that
        reads as the plainest of all is the one a constraint three statements above has already
        converted. `Binding.constraints` is therefore filed before this runs; see `_build_def_use`.

        A constraint the *named* binding carries is not one of these. It converted what that name
        was written with, at the write that carried it, and a read of the name hands over whatever
        it holds with nothing converting on the way out — measured, `[array]$x = 1, 2, 3; $y = $x;
        [Array]::Reverse($x); $y` writes `3 2 1`, the same as the script without the constraint.
        """
        for write in self._every_write():
            if write.may_define or not isinstance(write.node, Ps1Variable) or write.role.through:
                continue
            assignment = assignment_of(write.node)
            if assignment is None or assignment.operator != '=' or assignment.value is None:
                continue
            certain = True
            source: Node | None = assignment.value
            while source is not None and not isinstance(source, Ps1Variable):
                if isinstance(source, Ps1ParenExpression):
                    source = source.expression
                    continue
                if isinstance(source, Ps1CastExpression) or is_conversion_operator(source):
                    certain = False
                    source = _climbed_operand(source)
                    continue
                break
            if not isinstance(source, Ps1Variable):
                continue
            target = self._binding_of.get(id(write.node))
            named = self._binding_of.get(id(source))
            if target is None or named is None or target is named:
                continue
            yield Ps1AliasLink(write.node, target, named, certain and not target.constraints)

    def _every_write(self) -> Iterator[Occurrence]:
        for binding in self._every_binding():
            yield from binding.writes

    def _attribute_write(self, var: Ps1Variable, scope: Scope):
        binding = self._lookup_write_binding(var, scope)
        if binding is not None:
            self._record(binding, var, binding.writes)

    def _attribute_reference(self, var: Ps1Variable, scope: Scope):
        """
        Attribute a `[ref]$x` occurrence: resolved the way a read is, recorded the way a write is.

        The two halves are not the same question. PowerShell resolves the name by ordinary lookup,
        so a reference written inside a body reaches the enclosing binding and declares nothing —
        resolving it the way a write is resolved would look for a local binding that was never
        created and attribute the occurrence to nothing at all, losing the very use this exists to
        keep. What it then does to that binding is store into it, so it is recorded among the
        writes, where it both keeps the binding alive through `Binding.uses` and stops an earlier
        value reaching a later read.
        """
        if var.scope is not Ps1ScopeModifier.NONE:
            self._attribute_read(var, scope)
            return
        for binding in self._bindings_a_read_reaches(var, scope):
            self._record(binding, var, binding.writes)

    def _attribute_write_through(self, var: Ps1Variable, scope: Scope):
        """
        Attribute an occurrence a store reaches *through* — the `$x` of `$x[0] = 'z'`: resolved the
        way a read is, recorded the way a write is, and declaring nothing.

        Resolving it as a write would look for a binding in the scope the occurrence is written in
        and declare one PowerShell never creates, hiding the outer binding the store actually
        reaches. What it does to that binding is change the value under it, so it is recorded among
        the writes, where it kills whatever stood before it and leaves a read below it with a value
        nothing here can name.

        Unlike `[ref]`, a scope qualifier does not stop it. Measured: `[Array]::Reverse($script:x)`
        and `$script:x[0] = 9` both reach the script scope's array, so a qualified occurrence is
        recorded against the bindings the qualifier names rather than dropped to a read.

        It leaves `Binding.dynamic_or_qualified` alone, and so does every other write. That flag is
        what keeps a binding *no read names* alive, which is a question about reads; a qualified
        write of any kind — `$script:x = 5` as much as `$script:x[0] = 9` — is resolved through
        the scopes the qualifier names and needs nothing further.
        """
        for binding in self._bindings_a_read_reaches(var, scope):
            self._record(binding, var, binding.writes)

    def _bindings_a_read_reaches(self, var: Ps1Variable, scope: Scope) -> Iterator[Binding]:
        """
        Every binding an ordinary read of *var* written in *scope* could observe: the binding of the
        name in each enclosing scope for a bare reference, and the scopes `_qualified_read_scopes`
        names for a qualified one.

        Which one of several a reference resolves to depends on what ran, so every one of them is
        yielded and the caller records against all of them. For a write that is the conservative
        direction: a binding credited with a write it did not receive answers nothing about the
        values below it, where one that missed a write it did receive answers the value from before.
        """
        name = binding_key(var)
        if var.scope is Ps1ScopeModifier.NONE:
            cursor: Scope | None = scope
            while cursor is not None:
                binding = cursor.bindings.get(name)
                if binding is not None:
                    yield binding
                cursor = cursor.parent
            return
        for target in self._qualified_read_scopes(var, scope):
            binding = target.bindings.get(name)
            if binding is not None:
                yield binding

    def _attribute_named(self, cmd: Ps1CommandInvocation, scope: Scope):
        """
        File a command's string-addressed references against the bindings they name.

        A read resolves the way a bare variable read does, up the scope chain, and is recorded on
        every enclosing binding of the name — `Get-Variable x` inside a body observes whichever `$x`
        is in reach, and which one that is depends on what ran. A write resolves to the one scope
        the census placed it in.
        """
        for reference in named_references(cmd):
            role = NAME_ROLES[reference.role]
            if reference.role is Ps1NameRole.READS:
                cursor: Scope | None = scope
                while cursor is not None:
                    binding = cursor.bindings.get(reference.key)
                    if binding is not None:
                        binding.reads.append(
                            Occurrence(node=cmd, role=role, key=reference.key))
                    cursor = cursor.parent
                continue
            target = self._named_scope(reference, scope)
            if target is None:
                continue
            binding = target.bindings.get(reference.key)
            if binding is not None:
                binding.writes.append(Occurrence(node=cmd, role=role, key=reference.key))

    def _record(self, binding: Binding, var: Ps1Variable, into: list[Occurrence]) -> None:
        """
        File one variable occurrence against *binding*, and make it the occurrence's own binding
        unless an inner scope already claimed it — a bare reference is recorded on every enclosing
        binding of the name and resolves to the innermost.
        """
        into.append(Occurrence(node=var, role=occurrence_role(var), key=binding.name))
        self._binding_of.setdefault(id(var), binding)

    def _lookup_write_binding(self, var: Ps1Variable, scope: Scope) -> Binding | None:
        defining = self._defining_scope(var, scope)
        if defining is None:
            return None
        return defining.bindings.get(binding_key(var))

    def _attribute_read(self, var: Ps1Variable, scope: Scope):
        if var.scope is Ps1ScopeModifier.NONE:
            self._attribute_bare_read(var, scope)
        elif var.scope is Ps1ScopeModifier.ENV:
            binding = self.root_scope.bindings.get(binding_key(var))
            if binding is not None:
                self._record(binding, var, binding.reads)
        elif var.scope in _QUALIFIED_SCOPES:
            self._attribute_qualified_read(var, scope)

    def _attribute_qualified_read(self, var: Ps1Variable, scope: Scope):
        """
        Mark every binding a scope-qualified read can reach as `Binding.dynamic_or_qualified`, so it
        is never reported dead even though no occurrence in `Binding.reads` names it.
        """
        name = var.name.lower()
        primary: Binding | None = None
        for target in self._qualified_read_scopes(var, scope):
            binding = target.bindings.get(name)
            if binding is None:
                continue
            binding.dynamic_or_qualified = True
            if primary is None:
                primary = binding
        if primary is not None:
            self._binding_of[id(var)] = primary

    def _qualified_read_scopes(self, var: Ps1Variable, scope: Scope) -> Iterator[Scope]:
        """
        The scopes a scope-qualified read of *var* can reach. `$variable:` addresses the Variable
        provider drive, which resolves like a bare reference, so it reaches every enclosing scope;
        every other qualifier names the one scope `_defining_scope` binds a write through it in —
        the scope of the reference itself for `$local:` and `$private:`, the script scope for
        `$script:`, `$global:`, and `$using:`.
        """
        if var.scope is Ps1ScopeModifier.VARIABLE:
            cursor: Scope | None = scope
            while cursor is not None:
                yield cursor
                cursor = cursor.parent
            return
        defining = self._defining_scope(var, scope)
        if defining is not None:
            yield defining

    def _attribute_bare_read(self, var: Ps1Variable, scope: Scope):
        name = var.name.lower()
        cursor: Scope | None = scope
        while cursor is not None:
            binding = cursor.bindings.get(name)
            if binding is not None:
                self._record(binding, var, binding.reads)
            cursor = cursor.parent

Instance variables

var script_scope

The scope the script itself introduces — the outermost scope, whose bindings are the script-level variables.

Expand source code Browse git

@property
def script_scope(self) -> Scope:
    """
    The scope the script itself introduces — the outermost scope, whose bindings are the
    script-level variables.
    """
    return self.root_scope

var changes_an_object_in_place

Whether any occurrence in the script stores through a value rather than into a name — the $x of $x[0] = 9, of $h.k = 9 and of [Array]::Reverse($x).

A script with none never changes an object after it is built, so there a copy of one and a second name for it are indistinguishable, and a consumer weighing the two may stop asking. That is the question every sharing guard is really about, and it is a fact about the script rather than about the name a guard happens to be standing on: an object handed to a hashtable key, to a property or to a callee is changed under a name the guard cannot see.

Expand source code Browse git

@property
def changes_an_object_in_place(self) -> bool:
    """
    Whether any occurrence in the script stores *through* a value rather than into a name — the
    `$x` of `$x[0] = 9`, of `$h.k = 9` and of `[Array]::Reverse($x)`.

    A script with none never changes an object after it is built, so there a copy of one and a
    second name for it are indistinguishable, and a consumer weighing the two may stop asking.
    That is the question every sharing guard is really about, and it is a fact about the script
    rather than about the name a guard happens to be standing on: an object handed to a
    hashtable key, to a property or to a callee is changed under a name the guard cannot see.
    """
    if self._changes_in_place is None:
        self._changes_in_place = any(
            write.role.through for write in self._every_write())
    return self._changes_in_place

Methods

def scope_of(self, node)

The innermost scope that contains node, or None if the node was not part of the script the model was built from. A node in an if/loop/try body resolves to the enclosing script or scriptblock scope, since those bodies introduce no scope of their own.

Expand source code Browse git

def scope_of(self, node: Node) -> Scope | None:
    """
    The innermost scope that contains *node*, or `None` if the node was not part of the script
    the model was built from. A node in an `if`/loop/`try` body resolves to the enclosing script
    or scriptblock scope, since those bodies introduce no scope of their own.
    """
    return self._node_scope.get(id(node))

def binding_of(self, var)

The binding a variable occurrence resolves to — for a write, the binding in its defining scope; for a bare read, the nearest enclosing binding of the name — or None when the occurrence is free (an automatic or external variable the model never binds) or names a namespace outside the script's variables.

Expand source code Browse git

def binding_of(self, var: Ps1Variable) -> Binding | None:
    """
    The binding a variable occurrence resolves to — for a write, the binding in its defining
    scope; for a bare read, the nearest enclosing binding of the name — or `None` when the
    occurrence is free (an automatic or external variable the model never binds) or names a
    namespace outside the script's variables.
    """
    return self._binding_of.get(id(var))

def reads_in_scope(self, node, scope)

The names of scope's bindings read anywhere within node's subtree — every bare read of a name scope binds, including one nested in a scriptblock, but not the target of a plain = assignment, which replaces the value without observing it. A compound-assignment target ($x += 1) does observe it and counts as a read. This is the read set the dead-store sweep flushes pending stores against: unlike the walk it replaces, it does not stop at a nested scriptblock, so a store read only through a captured block is correctly seen as live.

Expand source code Browse git

def reads_in_scope(self, node: Node, scope: Scope) -> set[str]:
    """
    The names of *scope*'s bindings read anywhere within *node*'s subtree — every bare read of a
    name *scope* binds, including one nested in a scriptblock, but not the target of a plain `=`
    assignment, which replaces the value without observing it. A compound-assignment target
    (`$x += 1`) does observe it and counts as a read. This is the read set the dead-store sweep
    flushes pending stores against: unlike the walk it replaces, it does not stop at a nested
    scriptblock, so a store read only through a captured block is correctly seen as live.
    """
    names: set[str] = set()
    for descendant in node.walk():
        if not isinstance(descendant, Ps1Variable):
            continue
        if descendant.scope is not Ps1ScopeModifier.NONE:
            continue
        name = descendant.name.lower()
        if name in scope.bindings and not replaces_value(descendant):
            names.add(name)
    return names

def variables_in_scope(self, node, scope)

The names of scope's bindings referenced in any way — read or written — within node's subtree. The conservative flush set for a control-flow statement whose internal effect on a variable the linear sweep does not model: any mention of a bound name defers its pending store.

Expand source code Browse git

def variables_in_scope(self, node: Node, scope: Scope) -> set[str]:
    """
    The names of *scope*'s bindings referenced in any way — read or written — within *node*'s
    subtree. The conservative flush set for a control-flow statement whose internal effect on a
    variable the linear sweep does not model: any mention of a bound name defers its pending
    store.
    """
    names: set[str] = set()
    for descendant in node.walk():
        if isinstance(descendant, Ps1Variable) and descendant.scope is Ps1ScopeModifier.NONE:
            name = descendant.name.lower()
            if name in scope.bindings:
                names.add(name)
    return names

class Scope (kind, node, parent=None, children=<factory>, bindings=<factory>, writes_unreadable_names=False)

A lexical scope introduced by the script or a Ps1ScriptBlock. node is the introducing AST node, bindings maps a lowercased variable name to its Binding.

Expand source code Browse git

@dataclass(eq=False)
class Scope:
    """
    A lexical scope introduced by the script or a `refinery.lib.scripts.ps1.model.Ps1ScriptBlock`.
    `node` is the introducing AST node, `bindings` maps a lowercased variable name to its `Binding`.
    """
    kind: ScopeKind
    node: Node
    parent: Scope | None = None
    children: list[Scope] = field(default_factory=list)
    bindings: dict[str, Binding] = field(default_factory=dict)
    #: Whether a write this cannot place reaches every binding here: one aimed at the script scope
    #: from anywhere — `Set-Variable $n 'v' -Scope Global` — or at a scope the lexical chain cannot
    #: name at all, of which `-Scope 1` is the one that occurs. Every binding is then in doubt for
    #: as long as the tree stands, since the write may have landed on any of them and nothing says
    #: when.
    #:
    #: A write landing in the scope it is *written* in is not one of these. That one happens at a
    #: point, and `refinery.lib.scripts.ps1.analysis.dataflow.Ps1VariableFlow.unattributable_writes`
    #: holds it there, which leaves the reads before it answerable. Kept apart from
    #: `Binding.dynamic_or_qualified`, which says a *known* name is reachable another way; these are
    #: different reasons and a consumer may be able to live with one and not the other.
    writes_unreadable_names: bool = False

Instance variables

var kind

The type of the None singleton.

var node

The type of the None singleton.

var children

The type of the None singleton.

var bindings

The type of the None singleton.

var parent

The type of the None singleton.

var writes_unreadable_names

A write landing in the scope it is written in is not one of these. That one happens at a point, and Ps1VariableFlow.unattributable_writes() holds it there, which leaves the reads before it answerable. Kept apart from Binding.dynamic_or_qualified, which says a known name is reachable another way; these are different reasons and a consumer may be able to live with one and not the other.

class ScopeKind (*args, **kwds)

Create a collection of name/value pairs.

Example enumeration:

>>> class Color(Enum):
...     RED = 1
...     BLUE = 2
...     GREEN = 3

Access them by:

• attribute access:

Color.RED

• value lookup:

Color(1)

• name lookup:

Color['RED']

Enumerations can be iterated over, and know how many members they have:

>>> len(Color)
3

>>> list(Color)
[<Color.RED: 1>, <Color.BLUE: 2>, <Color.GREEN: 3>]

Methods can be added to enumerations, and members can have their own attributes – see the documentation for details.

Expand source code Browse git

class ScopeKind(enum.Enum):
    SCRIPT      = 'script'       # noqa
    FUNCTION    = 'function'     # noqa
    SCRIPTBLOCK = 'scriptblock'  # noqa

Ancestors

• enum.Enum

Class variables

var SCRIPT

The type of the None singleton.

var FUNCTION

The type of the None singleton.

var SCRIPTBLOCK

The type of the None singleton.

class StatementEffect (*args, **kwds)

The observable effect of evaluating a standalone statement, used by every pass that decides whether a statement can be pruned from a body:

• EFFECT: the statement performs a side effect (a command call, a store to a real variable, an increment); it must be preserved.
• OUTPUT: the statement is side-effect-free but yields a value to the enclosing pipeline (a bare constant, a pure expression); it is junk at a discarding position, but in a captured body it may be the return value, so removing it needs an emit-safety check.
• DISCARD: the statement is a syntactic no-op that yields nothing and does nothing observable (an empty statement, the $Null = <pure> and [Void]<pure> discard idioms, an Out-Null pipeline, a discarding ForEach); it is always safe to remove, even when it empties the body.

A discard idiom throws away a value, never the work that produced it: every one of them is recognized only over an operand that is_side_effect_free() accepts, so [Void](Start-Process x) is an EFFECT like any other call.

EFFECT deliberately says nothing about emission, and splitting it into an emitting and a silent member would not pay: a silent EFFECT is still un-removable, so every consumer would grow a branch to reach the verdict it already reaches. Write-Host x and Get-Item x are both EFFECT, and nothing here distinguishes them because nothing needs to.

Nor does any member say whether the statement can throw. That is expression_cannot_fault, which a caller about to remove an OUTPUT statement has to ask separately: [Int]'abc' and 1/0 are both OUTPUT, and removing either resumes a script that had terminated.

Expand source code Browse git

class StatementEffect(enum.Enum):
    """
    The observable effect of evaluating a standalone statement, used by every pass that decides
    whether a statement can be pruned from a body:

    - `EFFECT`: the statement performs a side effect (a command call, a store to a real variable, an
      increment); it must be preserved.
    - `OUTPUT`: the statement is side-effect-free but yields a value to the enclosing pipeline (a
      bare constant, a pure expression); it is junk at a discarding position, but in a captured body
      it may be the return value, so removing it needs an emit-safety check.
    - `DISCARD`: the statement is a syntactic no-op that yields nothing and does nothing observable
      (an empty statement, the `$Null = <pure>` and `[Void]<pure>` discard idioms, an `Out-Null`
      pipeline, a discarding `ForEach`); it is always safe to remove, even when it empties the body.

    A discard idiom throws away a *value*, never the work that produced it: every one of them is
    recognized only over an operand that `is_side_effect_free` accepts, so `[Void](Start-Process x)`
    is an `EFFECT` like any other call.

    `EFFECT` deliberately says nothing about emission, and splitting it into an emitting and a
    silent member would not pay: a silent `EFFECT` is still un-removable, so every consumer would
    grow a branch to reach the verdict it already reaches. `Write-Host x` and `Get-Item x` are both
    `EFFECT`, and nothing here distinguishes them because nothing needs to.

    Nor does any member say whether the statement can *throw*. That is `expression_cannot_fault`,
    which a caller about to remove an `OUTPUT` statement has to ask separately: `[Int]'abc'` and
    `1/0` are both `OUTPUT`, and removing either resumes a script that had terminated.
    """
    EFFECT = 'effect'
    OUTPUT = 'output'
    DISCARD = 'discard'

Ancestors

• enum.Enum

Class variables

var EFFECT

The type of the None singleton.

var OUTPUT

The type of the None singleton.

var DISCARD

The type of the None singleton.