Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DSC resource platform changes to use JSON instead of MOF #13359

Closed
SteveL-MSFT opened this issue Aug 5, 2020 · 20 comments
Closed

DSC resource platform changes to use JSON instead of MOF #13359

SteveL-MSFT opened this issue Aug 5, 2020 · 20 comments

Comments

@SteveL-MSFT
Copy link
Member

@SteveL-MSFT SteveL-MSFT commented Aug 5, 2020

Problem Statement

As we continue to make DSC more viable cross-platform, we have a serious problem with MOF. MOF parsing is provided by MMI.dll (there is both native code and managed code). MMI itself has dependency on OpenSSL 1.0 for the WSMan features. The non-Windows build of MMI is no longer supported and has been deprecated.

Proposal

To enable DSC resources to work cross-platform and on new distros that have moved to OpenSSL 1.1 (which is not backwards compatible with 1.0), we need to move away from MOF schema. This also means we would only support DSC resources written as PowerShell script (Get, Set, Test functions) or PowerShell classes. PowerShell classes already don't require a MOF schema, but many DSC Resources are script resources so we need to move to something other than MOF to describe the schema of that resource.

The current proposal is to move to JSON instead of MOF.

Engine Changes

System.Management.Automation.dll needs to be updated to understand JSON schema instead of MOF schema for PowerShell script DSC resources.

[NOTE!] Compilation of DSC configuration to MOF will not change for now, but in the future we also want to have configuration compile to a JSON format instead of MOF.

PSDesiredStateConfiguration module

This module exists outside of this repo. Plan is to eventually make it Open Source. For now, changes will need to be made to this module to work with JSON schema instead of MOF. In addition, it should expose a new cmdlet to help convert existing MOF schema to the new JSON schema format for existing DSC PowerShell script resources.

With this change, embedded objects passed to Get/Set/Test will be PSObjects instead of CimInstances. Unless the script is explicitly checking that it is a CimInstance, existing code should work with PSObject.

Also, a new Test-DscResourceSchema cmdlet will be added to validate the script resource matches the JSON schema.

What about LCM (Local Configuration Manager) and WindowsPowerShell?

Building upon the work in PS7.0 to enable use of Invoke-DscResource, the current plan is to not integrate with WindowsPowerShell LCM. This means that new DSC Resources based on JSON will not be compatible with WindowsPowerShell. This also means that existing DSC Resources that are WMI/OMI based or only have MOF schema will not be supported.

@iSazonov
Copy link
Collaborator

@iSazonov iSazonov commented Aug 6, 2020

Do you consider YAML instead of Json? It looks more user-friendly.

Do you plan use new .Net Json API for the project?

Should we have some RFC-s for the project?

With this change, embedded objects passed to Get/Set/Test will be PSObjects instead of CimInstances.

I feel strong-typed objects would be more convenient.
Especially if there is a scheme.

the current plan is to not integrate with WindowsPowerShell LCM.

Why? MSFT team invested in Windows Compatibility feature - call Windows PowerShell from PowerShell Core. They could do the same - call PS7 from Windows PS.

This means that new DSC Resources based on JSON will not be compatible with WindowsPowerShell.

Does this mean that Json schema is not compatible with MOF schema and it is inpossible to implement a simple converter?

This also means that existing DSC Resources that are WMI/OMI based or only have MOF schema will not be supported.

Where will not be supported in? In PS7? If so the same question about a schema converter.

@bergmeister
Copy link
Contributor

@bergmeister bergmeister commented Aug 6, 2020

Do you consider YAML instead of Json? It looks more user-friendly.

Since YAML is a superset of JSON, users will be able to convert to/from YAML at any time if they wish. The lack of an established standard library for YAML and PowerShell not shipping YAML support out of the box (and the lack of a community agreed module) is a strong argument against YAML IMHO

@cblackuk
Copy link

@cblackuk cblackuk commented Aug 6, 2020

This is FANTASTIC!!! Thank you for doing this! JSON/YAML FTW!!!

@bigoals365
Copy link

@bigoals365 bigoals365 commented Aug 6, 2020

I prefer YAMs over Mofs :}

Making YAML the "official" schema would be cleaner and more readable. JSON is format spaghetti.

Help make 2020 better with some common sense. Unless you really enjoy obfuscation :}

I now want to make a TiK Tok video about this... NOT

@johlju
Copy link

@johlju johlju commented Aug 6, 2020

Could it be made possible to have both MOF and JSON in the same DSC resource to make it compatible for both Windows PowerShell and PowerShell 7.1?

@Monte-Hazboun
Copy link

@Monte-Hazboun Monte-Hazboun commented Aug 6, 2020

I second johlju's request. For projects that use DSC and use both Win PowerShell and 7.1, having to maintain two separate versions of a resource module is a chore, even with a conversion tool. The ability to detect which version is being used and then load the correct format schema would be a huge boon.

I also second Bergmeister's comment and prefer JSON over YAML.

@SteveL-MSFT
Copy link
Member Author

@SteveL-MSFT SteveL-MSFT commented Aug 6, 2020

Do you consider YAML instead of Json? It looks more user-friendly.

The JSON is not intended to be used by users. It's a serialization format only.

I feel strong-typed objects would be more convenient.

The current is just a transformation from CimInstance to PSObject. Both are not strongly typed to the original object.

Why? MSFT team invested in Windows Compatibility feature - call Windows PowerShell from PowerShell Core. They could do the same - call PS7 from Windows PS.

The WindowsPowerShell LCM hosts .NET Framework. It would need lots of work to also host .NET Core. Our direction is "bring your own agent" where it could simply be a PowerShell script serving the same capabilities as LCM.

Existing WMI/OMI based DSC Resources are tied to legacy LCM which is not supported. Therefore, those specific type of resources are also not supported. We want to move towards a PowerShell script/classes based DSC resource model to enable faster development and also encourage cross platform instead of compiled code.

Could it be made possible to have both MOF and JSON in the same DSC resource to make it compatible for both Windows PowerShell and PowerShell 7.1?

Yes, the MOF will be ignored by the new DSC and the old DSC will ignore the JSON file.

We can discuss more of this at next DSC Community Call.

@SteveL-MSFT
Copy link
Member Author

@SteveL-MSFT SteveL-MSFT commented Aug 7, 2020

Do you plan use new .Net Json API for the project?

The changes to use JSON instead of MOF simply uses the ConvertTo-Json and ConvertFrom-Json cmdlets.

@iSazonov
Copy link
Collaborator

@iSazonov iSazonov commented Aug 7, 2020

The changes to use JSON instead of MOF simply uses the ConvertTo-Json and ConvertFrom-Json cmdlets.

If we want to move the cmdlets to .Net 5.0 Json API there will be many small breaking changes and makes sense to make this work earlier then later. Otherwise the new DSC project will catch the breaking changes after release and it is very bad.

@kvprasoon
Copy link
Contributor

@kvprasoon kvprasoon commented Aug 8, 2020

Our direction is "bring your own agent" where it could simply be a PowerShell script serving the same capabilities as LCM.

@SteveL-MSFT This is an awesome thought 👍

@iSazonov
Copy link
Collaborator

@iSazonov iSazonov commented Aug 10, 2020

Our direction is "bring your own agent" where it could simply be a PowerShell script serving the same capabilities as LCM.

It makes us think about some common module.

@Jeff-Jerousek
Copy link

@Jeff-Jerousek Jeff-Jerousek commented Aug 10, 2020

If the main argument for MOF was to support non-Windows builds and that support has been deprecated it makes sense to move it.

I hope the goal is the next 7 LTS release. Using the same configs irregardless of OS and letting the agent determine the details would be a strong feature to support moving to 7 in production environments.

@SteveL-MSFT
Copy link
Member Author

@SteveL-MSFT SteveL-MSFT commented Aug 10, 2020

I would love to see some LCM replacements as modules on PSGallery. Also, with "bring your own agent", it means integration with existing tools like Ansible, Puppet, and Chef should be easier as they can invoke DSC resources directly instead of going through LCM.

Goal is to have something people can start using in 7.1 and have a more complete story by 7.2 which is our next LTS.

@kgraves83
Copy link

@kgraves83 kgraves83 commented Aug 21, 2020

So if we are no longer using the built in LCM, would we use a model similar to Ansible, where we run a script from a workstation which connects to a server and runs a DSC module to apply the configuration? Would we still need to install the DSC modules on the server? Or could it be agentless and everything is installed on your workstation (like you can do with Ansible).

@SteveL-MSFT
Copy link
Member Author

@SteveL-MSFT SteveL-MSFT commented Aug 21, 2020

@kgraves83 with the current design, DSC modules will just be installed using PowerShellGet. This means they can be on a local fileshare or a remote gallery. So from that perspective, you could have orchestration initiated from a server, but everything else happens locally on the target (workstation).

@kvprasoon
Copy link
Contributor

@kvprasoon kvprasoon commented Aug 21, 2020

@SteveL-MSFT Feels like remotely triggering Test/Set/Get methods... How about the Pull server and LCM properties like ConfigurationMode etc... ? are all these left to people to design in the way anyone need ?

@gaelcolas
Copy link

@gaelcolas gaelcolas commented Sep 11, 2020

Given the challenges and the need to have a breaking change on all DSC resources using an embedded type (need to replace [CimInstance] in PSM1s by [PSObject], breaking the resource for PS5.1), I suggest dropping completely the MOF or JSON based DSC Resource, and focus on Class based resource, superior in many ways anyway.

See full comment here: #13399 (comment)

@wsmelton
Copy link

@wsmelton wsmelton commented Sep 22, 2020

Requiring a Class to be used would cut down the number of contributors you may have to DSC Resources. Writing PowerShell Classes has a harder entry point for many because it is not common (in beginner world at least) and most are used to writing function-style scripts and native PowerShell. Many have learned to code around JSON because it is commonly used with REST APIs.

@michaeltlombardi
Copy link

@michaeltlombardi michaeltlombardi commented Sep 23, 2020

From an integrating tool developer standpoint, JSON is much preferable to MOF for us. Right now, parsing resources for their API surface is less than pleasant; JSON would substantially ease that problem but not solve it (since it wouldn't aid with class-based resources).

@SteveL-MSFT
Copy link
Member Author

@SteveL-MSFT SteveL-MSFT commented Oct 1, 2020

Closing this in favor of #13731

@SteveL-MSFT SteveL-MSFT closed this Oct 1, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet