diff --git a/.gitattributes b/.gitattributes index a510f8d4..e0da1a88 100644 --- a/.gitattributes +++ b/.gitattributes @@ -2,3 +2,4 @@ * text eol=crlf *.exe -text *.zip -text +*.dll -text diff --git a/schema.json b/schema.json index a161f86e..430a8751 100644 --- a/schema.json +++ b/schema.json @@ -100,14 +100,23 @@ }, "type": { "enum": [ + "md5", "sha1", "sha256", "sha512" ] }, "url": { - "format": "uri", - "type": "string" + "anyOf": [ + { + "format": "uri", + "type": "string" + }, + { + "pattern": "^\\$url.[\\w\\d]+$", + "type": "string" + } + ] } }, "type": "object" @@ -190,8 +199,7 @@ "type": "string" }, "minItems": 1, - "type": "array", - "uniqueItems": true + "type": "array" } ] }, @@ -236,16 +244,21 @@ "anyOf": [ { "format": "uri", - "type": "string" + "type": "string", + "not": { + "pattern": "(\\$)" + } }, { "items": { "format": "uri", - "type": "string" + "type": "string", + "not": { + "pattern": "(\\$)" + } }, "minItems": 1, - "type": "array", - "uniqueItems": true + "type": "array" } ] } @@ -413,8 +426,29 @@ }, "version": { "type": "string" + }, + "suggest": { + "additionalProperties": false, + "patternProperties": { + "^(.*)$": { + "$ref": "#/definitions/stringOrArrayOfStrings" + } + }, + "type": "object" + }, + "psmodule": { + "additionalProperties": false, + "properties": { + "name": { + "type": "string" + } + }, + "type": "object" } }, "title": "scoop app manifest schema", - "type": "object" + "type": "object", + "required": [ + "version" + ] } diff --git a/supporting/validator/.gitignore b/supporting/validator/.gitignore new file mode 100644 index 00000000..23053de0 --- /dev/null +++ b/supporting/validator/.gitignore @@ -0,0 +1 @@ +packages/ diff --git a/supporting/validator/Newtonsoft.Json.Schema.dll b/supporting/validator/Newtonsoft.Json.Schema.dll new file mode 100644 index 00000000..9d13bce4 Binary files /dev/null and b/supporting/validator/Newtonsoft.Json.Schema.dll differ diff --git a/supporting/validator/Newtonsoft.Json.dll b/supporting/validator/Newtonsoft.Json.dll new file mode 100644 index 00000000..20dae627 Binary files /dev/null and b/supporting/validator/Newtonsoft.Json.dll differ diff --git a/supporting/validator/Scoop.Validator.cs b/supporting/validator/Scoop.Validator.cs new file mode 100644 index 00000000..2d75a076 --- /dev/null +++ b/supporting/validator/Scoop.Validator.cs @@ -0,0 +1,126 @@ +using System; +using System.Collections.Generic; +using System.IO; +using Newtonsoft.Json; +using Newtonsoft.Json.Linq; +using Newtonsoft.Json.Schema; + +namespace Scoop +{ + public class JsonParserException : Exception + { + public string FileName { get; set; } + public JsonParserException(string file, string message) : base(message) { this.FileName = file; } + public JsonParserException(string file, string message, Exception inner) : base(message, inner) { this.FileName = file; } + } + + public class Validator + { + private bool CI { get; set; } + public JSchema Schema { get; private set; } + public FileInfo SchemaFile { get; private set; } + public JObject Manifest { get; private set; } + public FileInfo ManifestFile { get; private set; } + public IList Errors { get; private set; } + public string ErrorsAsString + { + get + { + return String.Join(System.Environment.NewLine, this.Errors); + } + } + + private JSchema ParseSchema(string file) + { + try + { + return JSchema.Parse(File.ReadAllText(file, System.Text.Encoding.UTF8)); + } + catch (Newtonsoft.Json.JsonReaderException e) + { + throw new JsonParserException(Path.GetFileName(file), e.Message, e); + } + catch (FileNotFoundException e) + { + throw e; + } + } + + private JObject ParseManifest(string file) + { + try + { + return JObject.Parse(File.ReadAllText(file, System.Text.Encoding.UTF8)); + } + catch (Newtonsoft.Json.JsonReaderException e) + { + throw new JsonParserException(Path.GetFileName(file), e.Message, e); + } + catch (FileNotFoundException e) + { + throw e; + } + } + + public Validator(string schemaFile) + { + this.SchemaFile = new FileInfo(schemaFile); + this.Errors = new List(); + } + + public Validator(string schemaFile, bool ci) + { + this.SchemaFile = new FileInfo(schemaFile); + this.Errors = new List(); + this.CI = ci; + } + + public bool Validate(string file) + { + this.ManifestFile = new FileInfo(file); + return this.Validate(); + } + + public bool Validate() + { + this.Errors.Clear(); + try + { + if (this.Schema == null) + { + this.Schema = this.ParseSchema(this.SchemaFile.FullName); + } + this.Manifest = this.ParseManifest(this.ManifestFile.FullName); + } + catch (FileNotFoundException e) + { + this.Errors.Add(e.Message); + } + catch (JsonParserException e) + { + this.Errors.Add(String.Format("{0}{1}: {2}", (this.CI ? " [*] " : ""), e.FileName, e.Message)); + } + + if (this.Schema == null || this.Manifest == null) + return false; + + IList validationErrors = new List(); + + this.Manifest.IsValid(this.Schema, out validationErrors); + + if (validationErrors.Count > 0) + { + foreach (ValidationError error in validationErrors) + { + this.Errors.Add(String.Format("{0}{1}: {2}", (this.CI ? " [*] " : ""), this.ManifestFile.Name, error.Message)); + foreach (ValidationError childError in error.ChildErrors) + { + this.Errors.Add(String.Format((this.CI ? " [^] {0}{1}" : "{0}^ {1}"), new String(' ', this.ManifestFile.Name.Length + 2), childError.Message)); + } + } + } + + return (this.Errors.Count == 0); + } + } +} diff --git a/supporting/validator/Scoop.Validator.dll b/supporting/validator/Scoop.Validator.dll new file mode 100644 index 00000000..be157632 Binary files /dev/null and b/supporting/validator/Scoop.Validator.dll differ diff --git a/supporting/validator/build.ps1 b/supporting/validator/build.ps1 new file mode 100644 index 00000000..26e0161c --- /dev/null +++ b/supporting/validator/build.ps1 @@ -0,0 +1,8 @@ +$fwdir = gci C:\Windows\Microsoft.NET\Framework\ -dir | sort -desc | select -first 1 + +pushd $psscriptroot +& nuget restore -solutiondirectory . +gci $psscriptroot\packages\Newtonsoft.*\lib\net40\*.dll -file | % { copy-item $_ $psscriptroot } +& "$($fwdir.fullname)\csc.exe" /platform:anycpu /nologo /optimize /target:library /reference:Newtonsoft.Json.dll,Newtonsoft.Json.Schema.dll Scoop.Validator.cs +& "$($fwdir.fullname)\csc.exe" /platform:anycpu /nologo /optimize /target:exe /reference:Scoop.Validator.dll,Newtonsoft.Json.dll,Newtonsoft.Json.Schema.dll validator.cs +popd diff --git a/supporting/validator/packages.config b/supporting/validator/packages.config new file mode 100644 index 00000000..729db5cb --- /dev/null +++ b/supporting/validator/packages.config @@ -0,0 +1,5 @@ + + + + + diff --git a/supporting/validator/validator.cs b/supporting/validator/validator.cs new file mode 100644 index 00000000..287d8f6d --- /dev/null +++ b/supporting/validator/validator.cs @@ -0,0 +1,37 @@ +using System; +using System.IO; + +namespace Scoop +{ + public class Program + { + public static int Main(string[] args) + { + bool ci = (args.Length == 3 && args[2] == "-ci"); + bool valid = false; + + if (args.Length < 2) + { + Console.WriteLine("Usage: validator.exe schema.json manifest.json"); + return 1; + } + + Scoop.Validator validator = new Scoop.Validator(args[0], ci); + valid = validator.Validate(args[1]); + + if (valid) + { + Console.WriteLine("Yay! {0} validates against the schema!", Path.GetFileName(args[1])); + } + else + { + foreach (var error in validator.Errors) + { + Console.WriteLine(error); + } + } + + return valid ? 0 : 1; + } + } +} diff --git a/supporting/validator/validator.exe b/supporting/validator/validator.exe new file mode 100644 index 00000000..13ffadfd Binary files /dev/null and b/supporting/validator/validator.exe differ diff --git a/test/00-Project.Tests.ps1 b/test/00-Project.Tests.ps1 index f079a498..0b280192 100644 --- a/test/00-Project.Tests.ps1 +++ b/test/00-Project.Tests.ps1 @@ -4,7 +4,8 @@ $repo_files = @( Get-ChildItem $repo_dir -file -recurse -force ) $project_file_exclusions = @( $([regex]::Escape($repo_dir.fullname)+'\\.git\\.*$'), - '.sublime-workspace$' + '.sublime-workspace$', + 'supporting\\validator\\packages\\*' ) describe 'Project code' { @@ -78,7 +79,7 @@ describe 'Style constraints for non-binary project files' { # gather all files except '*.exe', '*.zip', or any .git repository files $repo_files | where-object { $_.fullname -inotmatch $($project_file_exclusions -join '|') } | - where-object { $_.fullname -inotmatch '(.exe|.zip)$' } + where-object { $_.fullname -inotmatch '(.exe|.zip|.dll)$' } ) $files_exist = ($files.Count -gt 0) diff --git a/test/Scoop-Manifest.Tests.ps1 b/test/Scoop-Manifest.Tests.ps1 index c0d23600..3ca15ee9 100644 --- a/test/Scoop-Manifest.Tests.ps1 +++ b/test/Scoop-Manifest.Tests.ps1 @@ -3,41 +3,68 @@ . "$psscriptroot\..\lib\manifest.ps1" describe "manifest-validation" { - $bucketdir = "$psscriptroot\..\bucket\" - $manifest_files = gci $bucketdir *.json - - $manifest_files | % { - it "test validity of $_" { - $manifest = parse_json $_.fullname - - $url = arch_specific "url" $manifest "32bit" - $url | should not match "\$" - $url64 = arch_specific "url" $manifest "64bit" - $url64 | should not match "\$" - if(!$url) { - $url = $url64 - } - $url | should not benullorempty - - $extract_dir32 = arch_specific "extract_dir" $manifest "32bit" - $extract_dir32 | should not match "\$" - $extract_dir64 = arch_specific "extract_dir" $manifest "64bit" - $extract_dir64 | should not match "\$" - - $manifest | should not benullorempty - $manifest.version | should not benullorempty - } - } -} - -describe "parse_json" { beforeall { - $working_dir = setup_working "parse_json" + $working_dir = setup_working "manifest" + $schema = "$psscriptroot\..\schema.json" + Add-Type -Path "$psscriptroot\..\supporting\validator\Newtonsoft.Json.dll" + Add-Type -Path "$psscriptroot\..\supporting\validator\Newtonsoft.Json.Schema.dll" + Add-Type -Path "$psscriptroot\..\supporting\validator\Scoop.Validator.dll" } - context "json is invalid" { + it "Scoop.Validator is available" { + ([System.Management.Automation.PSTypeName]'Scoop.Validator').Type | should be 'Scoop.Validator' + } + + context "parse_json function" { it "fails with invalid json" { - { parse_json "$working_dir\wget.json" } | should throw + { parse_json "$working_dir\broken_wget.json" } | should throw + } + } + + context "schema validation" { + it "fails with broken schema" { + $validator = new-object Scoop.Validator("$working_dir\broken_schema.json", $true) + $validator.Validate("$working_dir\wget.json") | should be $false + $validator.Errors.Count | should be 1 + $validator.Errors | select-object -First 1 | should belikeexactly "*broken_schema.json*Path 'type', line 6, position 4." + } + it "fails with broken manifest" { + $validator = new-object Scoop.Validator($schema, $true) + $validator.Validate("$working_dir\broken_wget.json") | should be $false + $validator.Errors.Count | should be 1 + $validator.Errors | select-object -First 1 | should belikeexactly "*broken_wget.json*Path 'version', line 5, position 4." + } + it "fails with invalid manifest" { + $validator = new-object Scoop.Validator($schema, $true) + $validator.Validate("$working_dir\invalid_wget.json") | should be $false + $validator.Errors.Count | should be 10 + $validator.Errors | select-object -First 1 | should belikeexactly "*invalid_wget.json*randomproperty*" + $validator.Errors | select-object -Last 1 | should belikeexactly "*invalid_wget.json*version." + } + } + + context "manifest validates against the schema" { + beforeall { + $bucketdir = "$psscriptroot\..\bucket\" + $manifest_files = gci $bucketdir *.json + $validator = new-object Scoop.Validator($schema, $true) + } + $manifest_files | % { + it "$_" { + $validator.Validate($_.fullname) + if ($validator.Errors.Count -gt 0) { + write-host -f yellow $validator.ErrorsAsString + } + $validator.Errors.Count | should be 0 + + $manifest = parse_json $_.fullname + $url = arch_specific "url" $manifest "32bit" + $url64 = arch_specific "url" $manifest "64bit" + if(!$url) { + $url = $url64 + } + $url | should not benullorempty + } } } } diff --git a/test/fixtures/manifest/broken_schema.json b/test/fixtures/manifest/broken_schema.json new file mode 100644 index 00000000..6ae6d5aa --- /dev/null +++ b/test/fixtures/manifest/broken_schema.json @@ -0,0 +1,11 @@ +{ + "$id": "http://scoop.sh/draft/schema#", + "$schema": "http://scoop.sh/draft/schema#", + "title": "scoop app manifest schema", + "type": "object" + "properties": { + "version": { + "type": "string" + } + } +} diff --git a/test/fixtures/parse_json/wget.json b/test/fixtures/manifest/broken_wget.json similarity index 100% rename from test/fixtures/parse_json/wget.json rename to test/fixtures/manifest/broken_wget.json diff --git a/test/fixtures/manifest/invalid_wget.json b/test/fixtures/manifest/invalid_wget.json new file mode 100644 index 00000000..9e10e7fc --- /dev/null +++ b/test/fixtures/manifest/invalid_wget.json @@ -0,0 +1,29 @@ +{ + "homepage": "https://eternallybored.org/misc/wget/", + "randomproperty": "should fail", + "license": "GPL3", + "architecture": { + "64bit": { + "url": [ + "https://eternallybored.org/misc/wget/wget-$version-win64.zip", + "http://curl.haxx.se/ca/cacert.pem" + ], + "hash": [ + "85e5393ffd473f7bec40b57637fd09b6808df86c06f846b6885b261a8acac8c5", + "" + ] + }, + "32bit": { + "url": [ + "https://eternallybored.org/misc/wget/wget-$version-win32.zip", + "http://curl.haxx.se/ca/cacert.pem" + ], + "hash": [ + "2ef82af3070abfdaf3862baff0bffdcb3c91c8d75e2f02c8720d90adb9d7a8f7", + "" + ] + } + }, + "bin": "wget.exe", + "post_install": "\"ca_certificate=$dir\\cacert.pem\" | out-file $dir\\wget.ini -encoding default" +} diff --git a/test/fixtures/manifest/wget.json b/test/fixtures/manifest/wget.json new file mode 100644 index 00000000..b6ebf32d --- /dev/null +++ b/test/fixtures/manifest/wget.json @@ -0,0 +1,29 @@ +{ + "homepage": "https://eternallybored.org/misc/wget/", + "license": "GPL3", + "version": "1.16.3", + "architecture": { + "64bit": { + "url": [ + "https://eternallybored.org/misc/wget/wget-1.16.3-win64.zip", + "http://curl.haxx.se/ca/cacert.pem" + ], + "hash": [ + "85e5393ffd473f7bec40b57637fd09b6808df86c06f846b6885b261a8acac8c5", + "" + ] + }, + "32bit": { + "url": [ + "https://eternallybored.org/misc/wget/wget-1.16.3-win32.zip", + "http://curl.haxx.se/ca/cacert.pem" + ], + "hash": [ + "2ef82af3070abfdaf3862baff0bffdcb3c91c8d75e2f02c8720d90adb9d7a8f7", + "" + ] + } + }, + "bin": "wget.exe", + "post_install": "\"ca_certificate=$dir\\cacert.pem\" | out-file $dir\\wget.ini -encoding default" +}