- Add preparing changes for extracting the main bucket to https://github.com/scoopinstaller/scoop-main
- Download main bucket on installation
- Add check_main_bucket function
- Add main bucket to buckets.json
- Add docker-compose.yml with default debugging image
- Remove old TODOs
- Refactor bucketdir -> Find-BucketDirectory
- Remove redundant .vscode settings
- Update supporting\validator\bin\Newtonsoft.Json to 11.0.2
- **HIGHLIGHT**: support for regex operator in JSON Paths
- Latest Newtonsoft.Json version is 12.0.1, which has some compatibility issues with Json.Schema 3.0.10 and PowerShell Core 6.1.12 (On my PC)
- Update supporting\validator\bin\Newtonsoft.Json.Schema to 3.0.10
- Use .NET 4.5 instead of .NET 4.0, since Scoop's requirements is [.NET Framework 4.5+](https://github.com/lukesampson/scoop#requirements)
- Modify `json_path()` to escape `$version` substitutions when `$jsonpath` has regex
**EXAMPLE OF JPATH REGEX**
```json
"checkver": {
"url": "https://dist.nuget.org/index.json",
"jp": "$..versions[?(@.displayName =~ /.*recommended.*/)].version"
}
```
- Enable output by setting $env:SCOOP_DEBUG to true (disabled by default)
- Converts objects to string output
- Shows calling position for clickable links in VSCode
reopen#3051
Now the code logic: if `SecurityProtocol` is `SystemDefault`, don't change, else change it to `Tls1.2,Tls1.1,Tls1.0`, never support `Ssl3` anymore.
* buckets: Move function into lib from lib-exec
There is no reason to have it inside lib-exec since, it's all internal functions, which could be used inside other code
lib-exec scripts should be slimmed as possible, as they are only code runners with some validation
* Fix whitespace
Fix a bug introduced in #2996 that single line hash extraction is broken.
When `find_hash_in_textfile()` is called in `get_hash_for_app()`, `$null` is used as `$regex` value for default single line hash extraction and the default value (`'^([a-fA-F0-9]+)$'`) is overriden, which leads a error of "hash not found".
Some JSON file ([example](https://mranapi.azurewebsites.net/api/download)) is surrounded by brackets ("[", "]") and cannot be parsed using `[Newtonsoft.Json.Linq.JObject]::Parse()` in `json_path()`. Instead, they could be parsed by `[Newtonsoft.Json.Linq.JArray]::Parse()` and then be treated as normal, e.g., applies a filter (script) expression.
- Add `[Newtonsoft.Json.Linq.JArray]::Parse()` after `[Newtonsoft.Json.Linq.JObject]::Parse()` failed
- Some JArray are handled by `json_path_legacy()` when `json_path()` throw error before, and the JSONPath is illegal when parsed with `JArray::Parse()`. To keep compatibility, add try-catch in this situation and let `json_path_legacy()` handle them still.
- In the above situation, manifest maintainer should use double dots after `$`, i.e., use `$..NODE` instead of `$.NODE` since it is a JArray contained the JObject.
**Data matters, please review**
This is a rework of #2891#2890 and #2882 . This will also fix#2724 and fix#2900 , close#2779
**Introduction**
I add a function called `unlink_persist_data($dir)`, which recursively unlink all junction/hard link in the given directory. This affects some sub-commands which have interaction with junction/hard link:
- `scoop-install`: the persisting logic has been improved as follow:
1. if there are data in the `persist` folder, also in app(`source`) folder, rename that one located in app folder (e.g. app's default setting files) with a `original` suffix, then create link from `persist` to app
2. if there are data in the `persist` folder, but no in app(`source`) folder, just create link from `persist` to app
3. if there is no data in the `persist` folder (e.g. fresh install), but there are data in app(`source`) folder (e.g. app's default setting files), we will just use that default setting files as the first-time persisting. So move that files from app folder to `persist` folder, then create link from `persist` to app
4. But what if if there is neither data in the `persist` folder (e.g. fresh install), nor in the app(`source`) folder (e.g. setting files will be created after first startup, like `Everthing.db`). We need to create empty persisting target in the `persist` folder. But by default we can't know if a persisting target is a file or a directory (e.g. `conf.d`). So we create a directory by default, and to avoid this, manifest maintainers should use `pre_install` to manually create the source file before persisting.
- `scoop-reset`: `reset` command uses the logic of `install`, but there is a problem: before re-persisting data, there have been `junction/hard link` in the app(`source`) folder. It will throw an exception of file exists #2724 . To fix this, we should unlink all old link before re-persisting, using `unlink_persist_data`.
- `scoop-uninstall`: `Remove-Item` can not remove `NTFS junction`, we need to unlink all persistting data, before uninstalling/deleting an app, but keeping persisting data.
- `scoop-cleanup`: like `uninstall`, `Remove-Item` can not remove `NTFS junction`, we need to unlink all persistting data, before deleting old versions of an app. Before PR #2882 , it use `fsutil.exe` to unlink junction, which is not friendly to restricted users (#2832 and #2881 ).
Beyond the logic improvement, there is a new feature now: it supports sub-folder data persisting, like:
```json
{
"homepage": "https://scoop.sh",
"description": "A dummy manifest for scoop tests.",
"license": "Freeware",
"version": "1.1.0",
"url": "https://get.scoop.sh",
"pre_install": [
"if (!(test-path \"$dir\\dummy.txt\")) { new-item -force \"$dir\\dummy.txt\" -itemtype file | out-null }",
"if (!(test-path \"$dir\\dummydir\")) { new-item \"$dir\\dummydir\" -itemtype directory | out-null }",
"if (!(test-path \"$dir\\subdir\")) { new-item \"$dir\\subdir\" -itemtype directory | out-null }",
"if (!(test-path \"$dir\\subdir\\subdummydir\")) { new-item \"$dir\\subdir\\subdummydir\" -itemtype directory | out-null }",
"if (!(test-path \"$dir\\subdir\\subdummy.txt\")) { new-item -force \"$dir\\subdir\\subdummy.txt\" -itemtype file | out-null }",
],
"persist": [
"dummy.txt",
"dummydir",
"subdir\\subdummydir",
"subdir\\subdummy.txt"
]
}
```
So no need to strip directories of source for target anymore.
To participate in the code review and tests, go visit https://github.com/h404bi/scoop-persist-test for test cases.
Chawye Hsu