Initial draft of configuration PS authoring experience

[Steve Lee (SteveL-MSFT)]

PR Summary

Initial draft proposing PS script authoring experience for configurations

[Gijs Reijn (Gijsreyn)]

Thanks Steve Lee (@SteveL-MSFT) for getting this started. I didn't fully catch you during the DSC WG meeting yesterday. Anyway, I like the shape of the proposal, especially its type object model, cmdlets for discovery, and scriptblocks transpiled to expressions. I think this is the right foundation to look at.

While going through the RFC, I collected these suggestions against the engine's implementation, so each one points at the relevant code you already defined in the RFC.

Schema handling in the proposed cmdlets

I reckon you might already have thought of something, but I'm just throwing it in here. Currently, the engine is quite opinionated about what a valid configuration document looks like. Every document should carry a $schema property that must match one of the recognized schema URIs, and all fields are camelcase - a resource instance containing unrecognized fields is rejected outright.

Parameters are not a list but an object keyed by parameter name, and whether a parameter is mandatory follows from whether it defines a defaultValue; there is no separate required concept in the schema. The document also versions itself through a first-class contentVersion property, and the Microsoft.DSC key inside metadata is reserved for engine-recognized settings such as the required security context.

This can be effectively addressed by having the proposed cmdlets own these details so authors never interact with them:

• New-DscConfiguration stamps $schema at creation - a simple -SchemaVersion value enum mapping to the recognized bundled URIs, with -SchemaUri as an override for engines newer than the module.
• When using New-DscConfiguration, a -ContentVersion parameter can map to contentVersion.
• The [Dsc.Configuration] types keep PascalCase properties in PowerShell and serialize to the schema's camelCase, so casing is the serializer's job.
• $config.Parameters is a keyed collection that serializes as the schema's map shape, keeping the +=/Add() experience from what you've already written.
• New-DscParameter -Required becomes validation sugar: it errors when combined with -DefaultValue and otherwise emits no default. The cmdlet also picks up the constraint fields the schema already defines: -AllowedValues, -MinValue/-MaxValue, -MinLength/-MaxLength, -Description, -Metadata.
• The reserved metadata namespace is a typed property, e.g. $config.Metadata.MicrosoftDsc.SecurityContext = [Dsc.SecurityContext]::Elevated, so engine-recognized settings are discoverable through intellisense.

Which results in the following document for the draft's example script:

{
  "$schema": "https://aka.ms/dsc/schemas/v3/bundled/config/document.json",
  "contentVersion": "1.0.0",
  "metadata": {
    "Name": "MyConfiguration",
    "Author": "SteveL-MSFT"
  },
  "parameters": {
    "computerName": {
      "type": "string"
    },
    "environment": {
      "type": "string",
      "defaultValue": "Production"
    }
  },
  "resources": [
    {
      "name": "My echo",
      "type": "Microsoft.DSC.Debug/Echo",
      "properties": {
        "output": "Hello World"
      }
    },
    {
      "name": "My echo 2",
      "type": "Microsoft.DSC.Debug/Echo",
      "properties": {
        "output": "[concat(parameters('environment'), ' ', parameters('computerName'))]"
      },
      "dependsOn": [
        "[resourceId('Microsoft.DSC.Debug/Echo', 'My echo')]"
      ]
    }
  ]
}

Covering the rest of the document schema

Also, the document model supports more than metadata, parameters, and resources. All of them fit into the proposed cmdlets:

Schema feature	Suggested authoring surface
variables	$config.Variables['x'] = ... (dictionary assignment; no cmdlet needed)
outputs (typed, with condition)	New-DscOutput -Name -Type -Value -Condition
User-defined functions (namespaced)	New-DscFunction -Namespace -Name -Parameter -OutputType -Value {scriptblock}
copy loops + copyIndex()	New-DscCopy -Name -Count -Mode Serial|Parallel -BatchSize, passed to New-DscResourceInstance -Copy
condition on resources	New-DscResourceInstance -Condition {scriptblock}
requireVersion (semver pinning)	New-DscResourceInstance -RequireVersion '>=1.2.0'
Nested resources	New-DscResourceInstance -Resources - needed anyway for groups and adapters
Resource directives	New-DscResourceInstance -RequireAdapter, -SecurityContext
Document directives	Typed $config.Directives (required engine version, discovery mode, security context)
Microsoft.DSC/Group, /Assertion, /Include	New-DscGroup, New-DscAssertion, New-DscInclude -Path -ParametersFilePath

Note

The table uses New-DscResourceInstance rather than the draft's New-DscResource. To PSDSC authors, New-DscResource already means "scaffold a new resource" (the resource designer's cmdlet), while this cmdlet declares a resource instance in a document - reusing the name would import the wrong muscle memory. New-DscResource could remain as an alias for brevity.

Build-time vs deploy-time semantics

I thought this was also worth mentioning, knowing that some of the functionality might get deprecated/removed in the long run. In the RFC, you mentioned looping. The engine provides these imperative constructs natively during deploy-time, meaning the copy loops with copyIndex(), condition on resources and outputs, if(), and higher-order functions like filter()/map(). Every imperative construct in an authoring script can have two plausible meanings in this case:

# Build-time: runs when the script runs, unrolled into N literal instances
foreach ($site in $sites) {
    $config.Resources.Add((New-DscResourceInstance -Type Microsoft.DSC.Debug/Echo -Name "site-$($site.name)" -Properties @{ output = $site.url }))
}

# Deploy-time: one instance with a copy block, expanded by the engine at apply time
$config.Resources.Add((New-DscResourceInstance -Type Microsoft.DSC.Debug/Echo -Name 'worker' `
    -Copy (New-DscCopy -Name workers -Count 3 -Mode Parallel) -Properties @{
        output = { "worker-$($dsc.CopyIndex('workers'))" }
    }))

Specifying which of these applies would prevent a lot of user surprise. I guess the following simple rule can apply:

1. Plain PowerShell is always build-time - loops, conditionals, and variables run when the script runs, so a foreach produces N literal instances.
2. Deploy-time is always explicit - a scriptblock property value, -Condition, -Copy, or a raw expression.
3. Additionally, PowerShell control flow is never implicitly promoted into copy/condition. For example:

# A PowerShell 'if' is evaluated now, at build time - depending on the machine
# generating the document, the resource is either in the document or it isn't
if ($env:BUILD_ENV -eq 'Production') {
   $config.Resources.Add((New-DscResourceInstance -Type Microsoft.DSC.Debug/Echo -Name 'audit' -Properties @{
       output = 'auditing on'
   }))
}

# The transpiler never converts that 'if' into a condition field on the author's
# behalf. To put the decision in the document and let the engine make it at
# apply time, say so explicitly:
$config.Resources.Add((New-DscResourceInstance -Type Microsoft.DSC.Debug/Echo -Name 'audit' `
   -Condition { $dsc.Parameters.environment -eq 'Production' } -Properties @{
       output = 'auditing on'
   }))

Note

With loops and conditionals now handled by the engine itself, what PowerShell authoring uniquely adds is everything that happens before the document exists: pulling context from inventories or APIs, reading input files, reusing logic across configurations, and producing a document per environment from a single script. Framing the Motivation section around that strength would also preempt the question of why this beats the schema-driven YAML completion VS Code already offers.

Transpiler: referencing model, operator mapping, and escaping

Loking at the scriptblock design, I came up with three additions:

1. A $dsc automatic variable. The expressions showcased now in the RFC require the transpiler to recognize member access on whatever the enclosing variable happens to be ($config.Parameters['Environment']). Injecting an automatic variable like $dsc into expression scriptblocks as the only deployment-time reference point makes the boundary more explicit and rename-safe:

# With current draft design
$echoResource2.Properties.Output = {
    $config.Parameters['Environment'] + ' ' + $config.Parameters['ComputerName']
}

# With proposed design
$echoResource2.Properties.Output = {
    $dsc.Parameters.environment + ' ' + $dsc.Parameters.computerName
}

• $dsc.Parameters.<name> -> parameters('<name>') (tab-completes from the document)
• $dsc.Variables.<name> -> variables('<name>')
• $dsc.Reference($instance) -> reference(resourceId(...)) - the most common cross-resource pattern, worth first-class support
• $dsc.CopyIndex('<loop>'), $dsc.Secret('<name>'), $dsc.EnvVar('<NAME>')
• $dsc.Fn.<functionName>(...) - a gateway generated from the engine's function registry (~80 functions), so new engine functions don't wait on transpiler updates; user-defined document functions appear as $dsc.Fn.'<namespace>.<name>'(...)

This gives the draft design well-defined semantics for anything that sits within the script block.

2. A normative operator mapping. The spec's "idiomatic PowerShell" bullet becomes a table the transpiler and users can rely on. Just some rough sketch examples (AI-generated):

# '+' maps by static operand type: numbers -> add(), strings/arrays -> concat();
# ambiguous operands are a build error with a fix-it suggesting $dsc.Fn.Add/Concat
{ $dsc.Parameters.retryCount + 1 }                # [add(parameters('retryCount'), 1)]
{ $dsc.Variables.prefix + '-web' }                # [concat(variables('prefix'), '-web')]

# Comparison and logic
{ $dsc.Parameters.environment -eq 'Production' }  # [equals(parameters('environment'), 'Production')]
{ $dsc.Parameters.count -gt 3 }                   # [greater(parameters('count'), 3)]
{ -not $dsc.Parameters.isProd }                   # [not(parameters('isProd'))]

# Null-coalescing and ternary
{ $dsc.Parameters.custom ?? 'default' }           # [coalesce(parameters('custom'), 'default')]
{ $dsc.Parameters.isProd ? 'prd' : 'dev' }        # [if(parameters('isProd'), 'prd', 'dev')]

# String interpolation, joining, and common string methods
{ "server-$($dsc.Parameters.environment)" }       # [format('server-{0}', parameters('environment'))]
{ $dsc.Parameters.tags -join ',' }                # [join(parameters('tags'), ',')]
{ $dsc.Parameters.name.ToLower() }                # [toLower(parameters('name'))]
{ $dsc.Parameters.name.StartsWith('web') }        # [startsWith(parameters('name'), 'web')]

# Pipelines over arrays
{ $dsc.Parameters.names | Where-Object { $_ -ne 'skip' } }    # [filter(parameters('names'), lambda(...))]
{ $dsc.Parameters.names | ForEach-Object { $_.ToUpper() } }   # [map(parameters('names'), lambda(...))]

Warning

Of course, this has its side effects. The spec needs a per-operator decision: either preserve PowerShell semantics in the emitted expression (e.g. -eq transpiles to equals(toLower(...), toLower(...))) or document the divergence and diagnose the risky cases at build time.

3. Escaping and escape hatch. The engine treats a leading [ as an expression and expects [[ for a literal. The cmdlets can make this safe by constructing. In this case, a value's type decides whether it is data or an expression, never its content:

# Plain strings are always literals. The exporter escapes a leading '[' so the
# engine never mistakes data for an expression - authors don't need to know the rule
$r.Properties.output = '[this is data, not an expression]'
# exports as: "output": "[[this is data, not an expression]"

# Scriptblocks are the ordinary way to produce an expression
$r.Properties.output = { $dsc.Parameters.computerName }
# exports as: "output": "[parameters('computerName')]"

# New-DscExpression emits a raw expression verbatim - the escape hatch for the
# window where the engine ships a function before the transpiler learns it
$r.Properties.output = New-DscExpression "newEngineFn(parameters('computerName'))"
# exports as: "output": "[newEngineFn(parameters('computerName'))]"

# Import reverses the escaping: "[[..." comes back as a plain string starting
# with '[', and "[...]" comes back as a [Dsc.Expression] - round-trips are lossless

Round-tripping, YAML, and validation

Before rounding it up with alternatives, I think it's worth having a round-tripping mechanism included. Why? Most of DSC's examples are all stored as *.dsc.yaml, with JSON as the interchange format. Export-DscConfiguration (and the Export() method) can infer format from the extension with a -Format override, and a ConvertTo- variant returning a string enables piping straight into the engine:

$config | ConvertTo-DscConfiguration -Format Json | dsc config test -f -

Bi-directional transpiling back to a script is rightly out of scope, but plain deserialization is going to be a cheap option and unlocks the most common brownfield workflows, e.g.: Import-DscConfiguration -Path old.dsc.yaml -> edit -> export.

It's also worth introducing a Test-DscConfiguration cmdlet (might run implicitly by export unless -SkipValidation?) can check:

1. The document validates against the JSON schema for its $schema URI.
2. type+name pairs are unique, including nested resources.
3. Every dependsOn target exists, and the dependency graph is acyclic.
4. Every expression parses against the engine grammar.
5. No secure-value literals appear anywhere in the document.
6. Properties conform to the cached resource schemas where available (a warning rather than an error when the type is unknown).

Lastly, the draft's IntelliSense builds on dsc resource list from the local machine. I think this can be tricky for cross-platform authoring and CI systems. Populating the cache on demand rather than at module import (adapter scans can be hella slow), plus Export-DscResourceCache/Import-DscResourceCache so a cache from a representative target can be checked into a repo and used on build agents, covers both. That way, you can easily warn rather than error on unknown resource types.

Alternate proposals worth listing

Plain simple, building forth on the example sketched out by Aditya Patwardhan (@adityapatwardhan) in the earlier discussion:

DSL sketch

$doc = DscConfiguration -SchemaVersion v3 {
    Parameter computerName -Type string -MinLength 1
    Parameter environment  -Type string -DefaultValue 'Production'
    Variable  greeting     -Value 'Hello'

    Resource 'My echo' -Type Microsoft.DSC.Debug/Echo -Properties @{
        output = 'Hello World'
    }

    Resource 'My echo 2' -Type Microsoft.DSC.Debug/Echo -DependsOn 'My echo' `
        -Condition { $dsc.Parameters.environment -eq 'Production' } -Properties @{
            output = { $dsc.Variables.greeting + ' ' + $dsc.Parameters.computerName }
        }

    foreach ($site in Get-Content ./sites.json | ConvertFrom-Json) {
        Resource "site-$($site.name)" -Type Microsoft.DSC.Debug/Echo -Properties @{ output = $site.url }
    }

    Output echoResult -Type string -Value { $dsc.Reference('My echo 2') }
}
$doc | Export-DscConfiguration -Path ./MyConfiguration.dsc.yaml

---

That's about it. Happy to turn any of the above into spec text. Again, thanks for getting this moving!

[Thomas Nieto (ThomasNieto)]

Shouldn't the namespace be something like Microsoft.Dsc.Parameter?

[Steve Lee (SteveL-MSFT)]

It should be, but I'm also trying to optimize for less typing

[Thomas Nieto (ThomasNieto)]

We could put a using statement at the top to reduce typing then the user wouldn't need to put in the name space at all.

[Ryan Yates (kilasuit)]

Overall generally agree with this proposal & can't wait to see what comes of this in future.

[Ryan Yates (kilasuit)]

Could we look at extending the how the Configuration Keyword works in a future version of PowerShell so that

• It outputs a configuration document in the new format by default not the older mof based documents

(This q is likely answered in the DscConfiguration Alternate proposal below)

[Ryan Yates (kilasuit)]

This all seems sensible enough to me

[Ryan Yates (kilasuit)]

I'd prefer to work this way as it aligns better with how DSC 1.1 works today however this feels like a We may get to this in future approach than a We plan to commit to this at this time

As mentioned above this likely ends up being a PowerShell Language change question to the Configuration Keyword though I do understand why that can of worms may want to be avoided.