lee/puter
1
0
Fork 0
mirror of https://github.com/HeyPuter/puter.git synced 2026-08-18 03:57:17 +00:00
Code Issues Actions 12 Packages Projects Releases Wiki Activity

Remove gross legacy tools and docs (#2873)

Browse Source
This commit is contained in:
Nariman Jelveh
2026-05-01 12:58:36 -07:00
committed by jelveh
parent 7cd762afa2
commit 60ea48dbaa
60 changed files with 31 additions and 5565 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitmodules
-12
View File
@@ -1,12 +0,0 @@
[submodule "submodules/v86"]
path = submodules/v86
url = git@github.com:HeyPuter/v86.git
[submodule "submodules/twisp"]
path = submodules/twisp
url = git@github.com:MercuryWorkshop/twisp.git
[submodule "submodules/epoxy-tls"]
path = submodules/epoxy-tls
url = git@github.com:MercuryWorkshop/epoxy-tls.git
[submodule "submodules/wiki"]
path = submodules/wiki
url = https://github.com/HeyPuter/puter.wiki.git
doc/AI.md
-38
View File
@@ -1,38 +0,0 @@
# Documentation for Robots
Hello, if you're an AI agent then you're reading the correct documentation.
Here are a few important notes:
- Puter is probably already cloned and configured, so avoid any setup
or configuration steps unless explicitly asked to perform them.
- Anything under `/src` (relative to the root of the repo) is probably
a workspace module. That means different directories might have different
code styles or use different import mechanisms (ESM vs CJS). Try to keep
changes consistent in the scope of where they are.
# Backend
Any file under `src/backend` that extends **BaseService** is called a
"backend service". Backend services can implement "traits". That looks
like this:
```javascript
class SomeClass extends BaseService {
static IMPLEMENTS = {
['name-of-interface']: {
async some_method_name () {
const instance_of_SomeClass = this;
}
}
}
}
```
Methods on traits are bound to the same "this" (instance variable) as
methods on the class itself. Trait methods cannot be indexed from the
instance variable; instead common functionality is usually moved to
regular instance methods which typically have an underscore at the end
of their name.
# Furher Documentation
Proceed to read the README.md document beside this file.
doc/File Structure.drawio
-214
View File
File diff suppressed because one or more lines are too long
doc/File Structure.drawio.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 187 KiB

doc/README.md
-45
View File
@@ -1,45 +0,0 @@
# Puter Documentation
Hi, you've found Puter's wiki page on GitHub! If you were looking for
something else, you might find it in the links below.
All of the wiki docs are generated from `doc/` directories in the main
repository, so it's best to edit docs there rather than here.
## Users
If you have general questions about using [Puter](https://puter.com),
our [community Discord](https://discord.gg/PQcx7Teh8u) and
[subreddit](https://www.reddit.com/r/puter/) are good places
to ask questions.
## Deployers
- [Hosting Instructions](./self-hosters/instructions.md)
- [Configuration](./self-hosters/config.md)
- [Domain Setup](./self-hosters/domains.md)
- [Support Levels](./self-hosters/support.md)
## App Developer Links
- [developer.puter.com](https://developer.puter.com)
- [docs.puter.com](https://docs.puter.com)
- share your apps on [Reddit](https://www.reddit.com/r/puter/) or
[Discord](https://discord.gg/PQcx7Teh8u)
## Contributor Documentation
### Where to Start
Start with [Repo Structure and Tooling](./contributors/structure.md).
### Index
- **Conventions**
- [Repo Structure and Tooling](./contributors/structure.md)
- How directories and files are organized in our GitHub repo
- What tools are used to build parts of Puter
- [Comment Prefixes](./contributors/comment_prefixes.md)
- A convention we use for line comments in code
- [Frontend Documentation](/src/gui/doc)
- [Backend Documentation](/src/backend/doc)
- [Extensions](./contributors/extensions/)
doc/contributors/comment_prefixes.md
-33
View File
@@ -1,33 +0,0 @@
# Comment Prefixes
Comments have prefixes using
[Conventional: Comments](https://conventionalcomments.org/)
as a **loose** guideline, and using this markdown file as a
the actual guideline.
This document will be updated on an _as-needed_ basis.
## The rules
- A comment line always looks like this:
- A whitespace character
- Optional prefix matching `/[a-z-]+\([a-z-]a+\):/`
- A whitespace character
- The comment
- Formalized prefixes must follow the rules below
- Any other prefix can be used. After some uses it
might be good to formalize it, but that's not a hard rule.
## Formalized prefixes
- `todo:` is interchangable with the famous `TODO:`, **except:**
when lowercase (`todo:`) it can include a scope: `todo(security):`.
- `track:` is used to track common patterns.
- Anything written after `track:` must be registered in
[track-comments.md](../devmeta/track-comments.md)
- `wet:` is usesd to track anything that doesn't adhere
to the DRY principle; the following message should describe
where similar code is
- `compare(<identifier>):` is used to note differences between other
implementations of a similar idea
- `name:` pedantic commentary on the name of something
doc/contributors/email_testing.md
-105
View File
@@ -1,105 +0,0 @@
# Local Email Testing
This guide describes how to set up and use [MailHog](https://github.com/mailhog/MailHog) for local email testing in Puter development. MailHog provides a local email server that captures outgoing emails for testing purposes without actually sending them to real recipients.
## Setup
### 1. Configure Puter
Add the following configuration to your `volatile/config/config.json` file:
```json
"email": {
"host": "localhost",
"port": 1025
}
```
### 2. Install MailHog
Download and run MailHog on your local machine:
```bash
# Install MailHog
wget https://github.com/mailhog/MailHog/releases/download/v1.0.1/MailHog_linux_amd64
chmod +x MailHog_linux_amd64
./MailHog_linux_amd64
```
### 3. Install Nodemailer
Install Nodemailer to send test emails to the SMTP server:
```bash
npm install nodemailer
```
## Using MailHog
### Access Web Interface
Once MailHog is running, access the web interface at:
[http://127.0.0.1:8025/](http://127.0.0.1:8025/)
All captured emails and their recipients will be displayed in this interface.
### Testing Your MailHog Setup with Nodemailer
You can verify that your MailHog instance is working correctly by creating a simple test script using Nodemailer. This allows you to send test emails that will be captured by MailHog without actually delivering them to real recipients.
Here's a sample script you can use to test your MailHog setup:
```javascript
import nodemailer from "nodemailer";
// Configure transporter to use MailHog
const transporter = nodemailer.createTransport({
host: "localhost", // MailHog SMTP server address
port: 1025, // Default MailHog SMTP port
secure: false // No SSL/TLS required for MailHog
});
// Define a test email
const mailOptions = {
from: "no-reply@example.com",
to: "test@example.com",
subject: "Hello from Nodemailer!",
text: "This is a test email sent using Nodemailer."
};
// Send the test email
transporter.sendMail(mailOptions)
.then(info => console.log("Email sent:", info.response))
.catch(error => console.error("Error:", error));
```
After sending an email with this script, you can view it in the MailHog web interface:
### How Puter Uses Nodemailer
Puter itself uses Nodemailer for sending emails through its `EmailService` class located in `/src/backend/src/services/EmailService.js`. This service handles various email templates for:
- Account verification
- Password recovery
- Two-factor authentication notifications
- File sharing notifications
- App approval notifications
- And more
The service creates a Nodemailer transport using the configuration from your `config.json` file, which is why setting up MailHog correctly is important for testing Puter's email functionality during development.
<img src="image.png" alt="Email in MailHog interface" width="300" height="200">
## Troubleshooting
If you encounter issues with MailHog:
1. Check if MailHog is running:
```bash
ps aux | grep MailHog
```
2. Ensure the correct port configurations in both MailHog and your application.
3. Check for any error messages in the MailHog console output.
doc/contributors/extensions.md
-38
View File
@@ -1,38 +0,0 @@
# Puter Extensions
## Quickstart
Create and edit this file: `mods/mods_enabled/hello-puter.js`
```javascript
const { UserActorType, AppUnderUserActorType } = use.core;
extension.get('/hello-puter', (req, res) => {
const actor = req.actor;
let who = 'unknown';
if ( actor.type instanceof UserActorType ) {
who = actor.type.user.username;
}
if ( actor.type instanceof AppUnderUserActorType ) {
who = actor.type.app.name + ' on behalf of ' + actor.type.user.username;
}
res.send(`Hello, ${who}!`);
});
```
## Events
//
This is subject to change as we make efforts to simplify the process.
### Step 1: Configure a Mod Directory
Add this to your config:
```json
"mod_directories": [
"{source}/../mods/mods_available"
]
```
This adds the `mods/mods_available` directory to this
doc/contributors/extensions/README.md
-89
View File
@@ -1,89 +0,0 @@
# Puter Extensions
## Quickstart
Create and edit this file: `mods/mods_enabled/hello-puter.js`
```javascript
// You can get definitions exposed by Puter via `use`
const { UserActorType, AppUnderUserActorType } = use.core;
// Endpoints can be registered directly on an extension
extension.get('/hello-puter', (req, res) => {
const actor = req.actor;
// Make a string "who" which says:
// "<username>", or:
// "<app> acting on behalf of <username>"
let who = 'unknown';
if ( actor.type instanceof UserActorType ) {
who = actor.type.user.username;
}
if ( actor.type instanceof AppUnderUserActorType ) {
who = actor.type.app.name
+ ' on behalf of '
+ actor.type.user.username;
}
res.send(`Hello, ${who}!`);
});
// Extensions can listen to events and manipulate Puter's behavior
extension.on('core.email.validate', event => {
if ( event.email.includes('evil') ) {
event.allow = false;
}
});
```
### Scope of `extension` and `use`
It is important to know that the `extension` global is temporary and does not
exist after your extension is loaded. If you wish to access the extension
object within a callback you will need to first bind it to a variable in
your extension's scope.
```javascript
const ext = extension;
extension.on('some-event', () => {
// This would throw an error
// extension.something();
// This works
ext.example();
})
```
The same is true for `use`. Calls to `use` should happen at the top of
the file, just like imports in ES6.
## Database Access
A database access object is provided to the extension via `extension.db`.
You **must** scope `extension` to another variable (`ext` in this example)
in order to access `db` from callbacks.
```javascript
const ext = extension;
extension.get('/user-count', { noauth: true, mw: [] }, (req, res) => {
const [count] = await ext.db.read(
'SELECT COUNT(*) as c FROM `user`'
);
});
```
The database access object has the following methods:
- `read(query, params)` - read from the database using a prepared statement. If read-replicas are enabled, this will use a replica.
- `write(query, params)` - write to the database using a prepared statement. If read-replicas are enabled, this will write to the primary.
- `pread(query, params)` - read from the database using a prepared statement. If read-replicas are enabled, this will read from the primary.
- `requireRead(query, params)` - read from the database using a prepared statement. If read-replicas are enabled, this will try reading from the replica first. If there are no results, a second attempt will be made on the primary.
## Events
See [events.md](./events.md)
## Definitions
See [definitions.md](./definitions.md)
doc/contributors/extensions/definitions.md
-14
View File
@@ -1,14 +0,0 @@
## Definitions
### `core.config` - Configuration
Puter's configuration object. This includes values from `config.json` or their
defaults, and computed values like `origin` and `api_origin`.
```javascript
const config = use('core.config');
extension.get('/get-origin', { noauth: true }, (req, res) => {
res.send(config.origin);
})
```
doc/contributors/extensions/gen.js
-38
View File
@@ -1,38 +0,0 @@
import dedent from 'dedent';
import events from './events.json.js';
const mdlib = {};
mdlib.h = (out, n, str) => {
out(`${'#'.repeat(n)} ${str}\n\n`);
};
const N_START = 3;
const out = str => process.stdout.write(str);
for ( const event of events ) {
mdlib.h(out, N_START, `\`${event.id}\``);
out(`${dedent(event.description) }\n\n`);
for ( const k in event.properties ) {
const prop = event.properties[k];
mdlib.h(out, N_START + 1, `Property \`${k}\``);
out(`${prop.summary }\n`);
out(`- **Type**: ${prop.type}\n`);
out(`- **Mutability**: ${prop.mutability}\n`);
if ( prop.notes ) {
out('- **Notes**:\n');
for ( const note of prop.notes ) {
out(` - ${note}\n`);
}
}
out('\n');
}
if ( event.example ) {
mdlib.h(out, N_START + 1, 'Example');
out(`\`\`\`${event.example.language}\n${dedent(event.example.code)}\n\`\`\`\n`);
}
out('\n');
}
doc/contributors/image.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 112 KiB

doc/contributors/structure.md
-64
View File
@@ -1,64 +0,0 @@
# Repository Structure and Tooling
Puter has many of its parts in a single [monorepo](https://en.wikipedia.org/wiki/Monorepo),
rather than a single repository for each cohesive part.
We feel this makes it easier for new contributors to develop Puter since you don't
need to figure out how to tie the parts together or how to work with Git submodules.
It also makes it easier for us to maintain project-wide conventions and tooling.
Some tools, like [puter-cli](https://github.com/HeyPuter/puter-cli), exist in separate
repositories. The `puter-cli` tool is used externally and can communicate with Puter's
API on our production (puter.com) instance or your own instance of Puter, so there's
not really any advantage to putting it in the monorepo.
## Top-Level directories
### The `doc` directory
The top-level `doc` directory contains the file you're reading right now.
Its scope is documentation for using and contributing to Puter in general,
and linking to more specific documentation in other places.
All `doc` directories will have a `README.md` which should be considered as
the index file for the documentation. All documentation under a `doc`
directory should be accessible via a path of links starting from `README.md`.
### The `src` directory
Every directory under `/tools` is [an npm "workspaces" module](https://docs.npmjs.com/cli/v8/using-npm/workspaces). Every direct child of this directory (generally) has a `package.json` and a `src` directory.
Some of these modules are core pieces of Puter:
- **Puter's backend** is [`/src/backend`](/src/backend)
- See [key locations in backend documentation](/src/backend/doc/contributors/structure.md)
- **Puter's GUI** is [`/src/gui`](/src/gui)
Some of these modules are apps:
- **Puter's Terminal**: [`/src/terminal`](/src/terminal)
- **Puter's Shell**: [`/src/phoenix`](/src/phoenix)
Some of these modules are libraries:
- **common javascript**: [`/src/putility`](/src/putility)
- **runtime import mechanism**: [`/src/useapi`](/src/useapi)
- **Puter's "puter.js" browser SDK**: [`/src/puter-js`](/src/puter-js)
### The `volatile` directory
When you're running Puter with development instructions (i.e. `npm start`),
Puter's configuration directory will be `volatile/config` and Puter's
runtime directory will be `volatile/runtime`, instead of the standard
`/etc/puter` and `/var/puter` directories in production installations.
We should probably rename this directory, actually, but it would inconvenience
a lot of people right now if we did.
### The `tools` directory
Every directory under `/tools` is [an npm "workspaces" module](https://docs.npmjs.com/cli/v8/using-npm/workspaces).
This is where `run-selfhosted.js` is. That's the entrypoint for `npm start`.
These tools are underdocumented and may not behave well if they're not executed
from the correct working directory (which is different for different tools).
Consider this a work-in-progress. If you want to use or contribute to anything
under this directory, for now you should
[tag @KernelDeimos on the community Discord](https://discord.gg/PQcx7Teh8u).
doc/contributors/vscode.md
-2
View File
@@ -1,2 +0,0 @@
### `vscode`
- `es6-string-html`
doc/devlog.md
-103
View File
@@ -1,103 +0,0 @@
## 2024-10-16
### Considerations for Mountpoints Feature
- `_storage_upload` takes paramter `uuid` instead of `path`
- S3 bucket strategy needs the UUID
- If we do hashes, 10MB chunks should be fine
- we're already able to smooth out bursty traffic using the
EWA algorithm
- Use of `systemFSEntryService`
- Is that normalized? Does everything go through this interface?
- Storage interface has methods like `post_insert`
- as far as I can tell this doesn't pose any issue
-
### Brainstorming Migration Strategies
#### Interface boundary at HL<->LL filesystem methods
-- **tags:** brainstorming
From the perspectice of a trait-oriented implementation,
which is not how LL/HL filesystem operations are currently implemented,
the LL-class operations are implemented in separate traits.
The composite trait containing all of these traits would be the trait
that represents a filesystem implementation itself.
Other filesystem interfaces that I've seen, such as FUSE and 9p,
all usually have a monolithic interface - that is to say, an interface
which includes all of the filesystem operations, rather than several
interfaces each implementing a single filesystem operaiton.
Something about the fact that the LL-class operations are in separate
classes makes it difficult to reason about how to move.
Is it simply that multiple files in a directory is just more
annoying to think about? Maybe, but there must be something more.
Perhaps it's that there are several references. Each implementation
(that is, implemenation of a single filesystem operation) could have
any number of different references across any number of different files.
This would not be the case with a monolithic interface.
I think the best of both worlds would be to have an interface representing
the entire filesystem and, in one place, link of of the individual
operation implementations to compose a filesystem implementation
### Filesystem Brainstorming
Puter's backend uses a service architecture. Each service is an instance
of a class extending "Service". A service can listen to events of the
backend's lifecycle, interact with other services, and interact with
external interfaces such as APIs and databases.
Puter's current filesystem, let's call it PuterFSv1, exists as the result
of multiple services working together. We have LocalDiskStorageService
which mimics an S3 bucket on a local system, and we have
DatabaseFSEntryService which manages information about files, directories,
and their relationships within the database, and therefore depends on
DatabaseAccessService.
It is now time to introduce a MountpointService. This will allow another
service or a user's configuration to assign an instance of a filesystem
implementation (such as PuterFSv1) to a specific path.
The trouble here is that PuterFSv1 is composed of services, and the nature
of a service is such that it exists for the lifecycle of the application.
The class for a particular service can be re-used and registered with
multiple names (creating multiple services with the same implementation
but perhaps different configuration), but that's only a clean scenario when
there is just one service. PuterFSv1, on the other hand, is like an
imaginary service composed of other services.
The following possibilities then should be discussed:
- CompositeService base class for a service that is composed of
more than one service.
- Refactor filesystem to not use service architecture.
- Each filesystem service can manage state and configuration
for multiple mountpoints
(I don't like this idea; it feels messy. I wonder what software
principles this violates)
We can take advantage of traits/interfaces here.
PuterFSv1 depends on two interfaces:
- An S3-like data storage implementation
- An fsentry storage implementation
Counterintuitively from what I first thought, "Refactor the filesystem"
actually looks like the best solution, and it doens't even look like it
will be that difficult. In fact, it'll likely make the filesystem easier
to maintain and more robust as a result.
Additionally, we can introduce PuterFSv2, which will introduce storing
data in chunks identified by their hashes, and associated hashes with
fsentries.
PuterFSService will be a new service which registers 'PuterFSv1' with
FilesystemService.
An instance of a filesystem needs to be separate from a mountpoint.
For example, PuterFSv1 will usually have only one instance but it may
be mounted several different times. `/some-user` on Puter's VFS could
be a mountpoint for `/some-user` in the instance of PuterFSv1.
doc/devmeta/track-comments.md
-62
View File
@@ -1,62 +0,0 @@
# Track Comments
Comments beginning with `// track:`. See
[comment_prefixes.md](../contributors/comment_prefixes.md)
## Track Comment Registry
- `track: type check`:
A condition that's used to check the type of an imput.
- `track: adapt`
A value can by adapted from another type at this line.
- `track: bounds check`:
A condition that's used to check the bounds of an array
or other list-like entity.
- `track: ruleset`
A series of conditions that early-return or `continue`
- `track: object description in comment`
A comment above the creation of some object which
could potentially have a `description` property.
This is especially relevant if the object is stored
in some kind of registry where multiple objects
could be listed in the console.
- `track: slice a prefix`
A common pattern where a prefix string is "sliced off"
of another string to obtain a significant value, such
as an indentifier.
- `track: actor type`
The sub-type of an Actor object is checked.
- `track: scoping iife`
An immediately-invoked function expression specifically
used to reduce scope clutter.
- `track: good candidate for sequence`
Some code involves a series of similar steps,
or there's a common behavior that should happen
in between. The Sequence class is good for this so
it might be a worthy migration.
- `track: opposite condition of sibling`
A sibling class, function, method, or other construct of
source code has a boolean expression which always evaluates
to the opposite of the one below this track comment.
- `track: null check before processing`
An object could be undefined or null, additional processing
occurs after a null check, and the unprocessed object is not
relevant to the rest of the code. If the code for obtaining
the object and processing it is moved to a function outside,
then the null check should result in a early return of null;
this code with the track comment may have additional logic
for the null/undefined case.
- `track: manual safe object`
This code manually creates a new "client-safe" version of
some object that's in scope. This could be either to pass
onto the browser or to pass to something like the
notification service.
- `track: common operations on multiple items`
A patterm which emerges when multiple variables have
common operations done upon them in sequence.
It may be applicable to write an iterator in the
future, or something will come up that require
these to be handled with a modular approach instead.
- `track: checkpoint`
A location where some statement about the state of the
software must hold true.
doc/docmeta.md
-45
View File
@@ -1,45 +0,0 @@
# Meta Documentation
Guidelines for documentation.
## How documentation is organized
This documentation exists in the Puter repository.
You may be reading this on the GitHub wiki instead, which we generate
from the repository docs. These docs are always under a directory
named `doc/`.
From [./contributors/structure.md](./contributors/structure.md):
> The top-level `doc` directory contains the file you're reading right now.
> Its scope is documentation for using and contributing to Puter in general,
> and linking to more specific documentation in other places.
>
> All `doc` directories will have a `README.md` which should be considered as
> the index file for the documentation. All documentation under a `doc`
> directory should be accessible via a path of links starting from `README.md`.
### Documentation Structure
The top-level `doc` directory contains the following subdirectories:
- `api/` - API documentation for Puter services
- `contributors/` - Documentation for contributors to the Puter project
- `devmeta/` - Meta documentation for developers
- `i18n/` - Internationalization documentation
- `planning/` - Project planning documentation
- `self-hosters/` - Documentation for self-hosting Puter
- `uncategorized/` - Miscellaneous documentation
As well as some files:
- `README.md` - Documentation overview optimized for humans.
- `AI.md` - Documentation overview optimized for AI/LLM agents.
Module-specific documentation follows a similar structure, with each module having its own `doc` directory. For contributor-specific documentation within a module, use a `contributors` subdirectory within the module's `doc` directory.
## Docs Styleguide
### "is" and "is not"
- When "A is B", bold "is": "A **is** B" (`A **is** B`)
- When "A is not B", bold "not": "A is **not** B" (`A is **not** B`)
doc/license_header.txt
+1 -1
View File
@@ -1,4 +1,4 @@
Copyright (C) 2024 Puter Technologies Inc.
Copyright (C) 2024-present Puter Technologies Inc.
This file is part of Puter.
doc/prod.md
-149
View File
@@ -1,149 +0,0 @@
# Puter in Production
## Building
```bash
npm run build
```
## Usage
Will build Puter in the `dist` directory. Include the generated `./dist/gui.js` file in your HTML page and call `gui()` when the page is loaded:
```html
<script type="text/javascript" src="./dist/gui.js"></script>
<script type="text/javascript">
window.addEventListener('load', function() {
// Initialize the GUI. All options are optional!
gui({
// The origin of the app. This is the base URL of the GUI.
gui_origin: "https://puter.com",
// The origin of the API. This is the base URL of the API endpoints that the GUI will call for all its operations.
api_origin: "https://api.puter.com",
// The domain under which user websites are hosted.
hosting_domain: "puter.site",
// The maximum length of file/directory names.
max_item_name_length: 500,
// If GUI has to enforce email verification before allowing user to publish a website.
require_email_verification_to_publish_website: true,
})
});
</script>
```
## Full Production Example
Assuming the following directory structure in production:
```
.
├── dist/
│ ├── favicons/
│ ├── images/
│ ├── bundle.min.css
│ ├── bundle.min.js
│ ├── gui.js
│ └── ...
└── index.html
```
The `index.html` file below will load Puter and all the necessary meta tags, favicons, and branding assets:
```html
<!DOCTYPE html>
<html lang="en">
<head>
<title>Puter</title>
<meta name="author" content="Puter Technologies Inc.">
<meta name="description" content="Puter is a privacy-first personal cloud to keep all your files, apps, and games in one private and secure place, accessible from anywhere at any time.">
<meta name="facebook-domain-verification" content="e29w3hjbnnnypf4kzk2cewcdaxym1y" />
<link rel="canonical" href="https://puter.com">
<!-- Meta meta tags -->
<meta property="og:url" content="https://puter.com">
<meta property="og:type" content="website">
<meta property="og:title" content="Puter">
<meta property="og:description" content="Puter is a privacy-first personal cloud to keep all your files, apps, and games in one private and secure place, accessible from anywhere at any time.">
<meta property="og:image" content="./dist/images/screenshot.png">
<!-- Twitter meta tags -->
<meta name="twitter:card" content="summary_large_image">
<meta property="twitter:domain" content="puter.com">
<meta property="twitter:url" content="https://puter.com">
<meta name="twitter:title" content="Puter">
<meta name="twitter:description" content="Puter is a privacy-first personal cloud to keep all your files, apps, and games in one private and secure place, accessible from anywhere at any time.">
<meta name="twitter:image" content="./dist/images/screenshot.png">
<!-- favicons -->
<link rel="apple-touch-icon" sizes="57x57" href="./dist/favicons/apple-icon-57x57.png">
<link rel="apple-touch-icon" sizes="60x60" href="./dist/favicons/apple-icon-60x60.png">
<link rel="apple-touch-icon" sizes="72x72" href="./dist/favicons/apple-icon-72x72.png">
<link rel="apple-touch-icon" sizes="76x76" href="./dist/favicons/apple-icon-76x76.png">
<link rel="apple-touch-icon" sizes="114x114" href="./dist/favicons/apple-icon-114x114.png">
<link rel="apple-touch-icon" sizes="120x120" href="./dist/favicons/apple-icon-120x120.png">
<link rel="apple-touch-icon" sizes="144x144" href="./dist/favicons/apple-icon-144x144.png">
<link rel="apple-touch-icon" sizes="152x152" href="./dist/favicons/apple-icon-152x152.png">
<link rel="apple-touch-icon" sizes="180x180" href="./dist/favicons/apple-icon-180x180.png">
<link rel="icon" type="image/png" sizes="192x192" href="./dist/favicons/android-icon-192x192.png">
<link rel="icon" type="image/png" sizes="32x32" href="./dist/favicons/favicon-32x32.png">
<link rel="icon" type="image/png" sizes="96x96" href="./dist/favicons/favicon-96x96.png">
<link rel="icon" type="image/png" sizes="16x16" href="./dist/favicons/favicon-16x16.png">
<link rel="manifest" href="./dist/manifest.json">
<meta name="msapplication-TileColor" content="#ffffff">
<meta name="msapplication-TileImage" content="./dist/favicons/ms-icon-144x144.png">
<meta name="theme-color" content="#ffffff">
<!-- Preload images when applicable -->
<link rel="preload" as="image" href="./dist/images/wallpaper.webp">
</head>
<body>
<!-- Load the GUI script -->
<script type="text/javascript" src="./dist/gui.js"></script>
<!-- Initialize GUI when document is loaded -->
<script type="text/javascript">
window.addEventListener('load', function() {
gui()
});
</script>
</body>
</html>
```
### Server settings
The GUI is a single page application (SPA) and as best practice any route under root (`/*`) should preferably load the `index.html` file. However, there are situations where we want to load a custom page for a specific route: for example, the `/privacy` route may need to load a page that contains your privacy policy and has nothing to do with the GUI application. In these cases it is ok to load a custom page as long as the following essential GUI routes are loaded with the GUI (i.e. `index.html` file):
- `/app/*`
- `/action/*`
In other words, consider the routes above as "reserved" for Puter.
### Publish My Website
Right-click anywhere on the desktop to display options
From the options menu, select "New".
Then, choose "Folder".
Give the folder a name according to your preference.
After creating the folder:
Right-click on the folder.
Select the option "Publish as Website".
### Best Practices
- The `title` tags and meta tags (`<title></title>`, `<meta property="og:title"`, `<meta name="twitter:title"`, ...) should be dynamically set by the server. For example, if the URL is of an app (e.g. `https://puter.com/app/editor`) the `title` tags and meta tags should contain the app's title rather than the generic Puter title.
- The `description` meta tags (`<meta name="description"`, `<meta property="og:description"`, `<meta name="twitter:description"`, ...) should be dynamically set by the server. For example, if the URL is of an app (e.g. `https://puter.com/app/editor`) the `description` meta tags should contain the app's description rather than the generic Puter description.
- Make sure to escape any HTML code that is dynamically added to the HTML page. For example, if the app's description is `Puter is a <b>privacy-first</b> personal cloud to keep all your files, apps, and games in one private and secure place, accessible from anywhere at any time.` the `<b>` tag should be escaped to `&lt;b&gt;` so that the browser doesn't interpret it as an HTML tag.
- Make sure to replace all new line characters with space when dynamically adding text to the HTML page.
- Generally, for UX and SEO reasons make sure that the tags are filled with relevant information about the state the URL is representing. E.g. is the user on the desktop or an app?
doc/self-hosters/config-vals.json.js
-84
View File
@@ -1,84 +0,0 @@
export default [
{
key: 'domain',
description: `
Domain name of the Puter instance. This may be used to generate URLs
in the UI. If "allow_all_host_values" is false or undefined, the domain
will be used to validate the host header of incoming requests.
`,
example_values: [
'example.com',
'subdomain.example.com',
],
},
{
key: 'protocol',
description: `
The protocol to use for URLs. This should be either "http" or "https".
`,
example_values: [
'http',
'https',
],
},
{
key: 'static_hosting_domain',
description: `
This domain name will be used for public site URLs. For example: when
you right-click a directory and choose "Publish as Website".
This domain should point to the same server. If you have a LAN configuration
you could set this to something like
\`site.192.168.555.12.nip.io\`, replacing
\`192.168.555.12\` with a valid IP address belonging to the server.
`,
},
{
key: 'allow_all_host_values',
description: `
If true, Puter will accept any host header value in incoming requests.
This is useful for development, but should be disabled in production.
`,
},
{
key: 'allow_nipio_domains',
description: `
If true, Puter will allow requests with host headers that end in nip.io.
This is useful for development, LAN, and VPN configurations.
`,
},
{
key: 'http_port',
description: `
The port to listen on for HTTP requests.
`,
},
{
key: 'enable_public_folders',
description: `
If true, any /username/Public directory will be available to all
users, including anonymous users.
`,
},
{
key: 'disable_temp_users',
description: `
If true, new users will see the login/signup page instead of being
automatically logged in as a temporary user.
`,
},
{
key: 'disable_user_signup',
description: `
If true, the signup page will be disabled and the backend will not
accept new user registrations.
`,
},
{
key: 'disable_fallback_mechanisms',
description: `
A general setting to prevent any fallback behavior that might
"hide" errors. It is recommended to set this to true when
debugging, testing, or developing new features.
`,
},
];
doc/self-hosters/first-run-issues.md
-80
View File
@@ -1,80 +0,0 @@
# First Run Issues
## "Cannot find package '@heyputer/backend'"
Scenario: You see the following output:
```
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Cannot find package '@heyputer/backend' ┃
┃ 📝 this usually happens if you forget `npm install` ┃
┃ Suggestions: ┃
┃ - try running `npm install` ┃
┃ Technical Notes: ┃
┃ - @heyputer/backend is in an npm workspace ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
```
1. Ensure you have run `npm install`.
2. [Install build essentials for your distro](#installing-build-essentials),
then run `npm install` again.
## Installing Build Essentials
### Debian-based distros
```
sudo apt update
sudo apt install build-essential
```
### RHEL-family distros (Fedora, Rocky, etc)
For distros using dnf5 (Fedora 41+):
```
sudo dnf install @development-tools
```
Otherwise:
```
sudo dnf groupinstall "Development Tools"
```
### "I use Arch btw"
```
sudo pacman -S base-devel
```
### Alpine
If you're running in Puter's Alpine image then this is already installed.
```
sudo apk add build-base
```
### Gentoo
You know what you're doing; you just wanted to see if we mentioned Gentoo.
## "Could not load the "sharp" module using the freebsd-x64 runtime"
In order to get it to work on FreeBSD, you will need to build sharp from source and link it to the project.
```
pkg install vips
git clone --depth=1 https://github.com/lovell/sharp.git
cd sharp
yarn install
sudo npm link
```
After `npm install` you can link the prebuilt module
```
# cd puter
# npm install
npm link sharp
npm start
```
doc/self-hosters/gen.js
-25
View File
@@ -1,25 +0,0 @@
import dedent from 'dedent';
import configVals from './config-vals.json.js';
const mdlib = {};
mdlib.h = (out, n, str) => {
out(`${'#'.repeat(n)} ${str}\n\n`);
};
const N_START = 3;
const out = str => process.stdout.write(str);
for ( const configVal of configVals ) {
mdlib.h(out, N_START, `\`${configVal.key}\``);
out(`${dedent(configVal.description) }\n\n`);
if ( configVal.example_values ) {
mdlib.h(out, N_START + 1, 'Examples');
for ( const example of configVal.example_values ) {
out(`- \`"${configVal.key}": ${JSON.stringify(example)}\`\n`);
}
}
out('\n');
}
doc/self-hosters/support.md
-39
View File
@@ -1,39 +0,0 @@
## Puter Support Levels for Repository Updates
This document describes issues requiring repository changes;
which issues will be fixed by Puter's core team, and which ones
will be fixed if the community makes a contribution.
This document is not "law". It is provided only as a helpful guide
on what to expect.
### Level Glossary
| Name | Description |
| ---- | ----------- |
| Core | Core developers will fix this |
| Community | We will accept contributions to fix this |
| Mixed | Core developers will fix this if it's currently a priority |
### Issues and their Levels
| Issue | Priority |
| ----- | -------- |
| Security vulnerability | Core |
| Breaking change to SDK or API | Core |
| Bug in service in CoreModule | Core |
| Bug in a built-in app | Core |
| Login/init failure in Docker on `release` branch | Core |
| Login/init failure in Linux or OSX | Core |
| Login/init failure in Docker on `main` branch | Mixed |
| Login/init failure with specific configuration | Mixed |
| Login/init failure in Windows | Community |
## Puter Support for a Particular Deployment
If you experience issues on a self-hosted deployment we're here to
help. Some issues are related to configuration or environment, so
we may only be able to help in a limited capacity. Issues related
to data loss, data corruption, or security will have higher priority
over other issues with particular deployments.
doc/testing_with_email.md
-31
View File
@@ -1,31 +0,0 @@
# Testing with Email
Testing anything involving email is really simple using [mailhog](https://github.com/mailhog/MailHog)
### Step 1: Configure email service
In your `config.json` for Puter (`volatile/config/config.json` usually, `/var/puter/config.json` in containers),
add this entry to the `"services`" map:
```javascript
"services": {
// ... there are probably other service configs
"email": {
"host": "localhost",
"port": 1025
}
}
```
### Step 2: Install and run mailhog
Follow the instructions on [MailHog](https://github.com/mailhog/MailHog)'s
repository, or install through your distro's package manager.
Run the command: `mailhog`.
You should now have an inbox at [http://127.0.0.1:8025](http://127.0.0.1:8025).
Every email that Puter sends will show up on this page.
eslint.config.js
-1
View File
@@ -98,7 +98,6 @@ export default defineConfig([
'src/gui/**',
'src/docs/**',
'src/puter-js/**',
'src/useapi/**',
'src/worker/**',
'submodules/**',
'tests/**',
install.md
-139
View File
@@ -1,139 +0,0 @@
# INSTALL.md
## Node.js & npm Installation Guide
## 1. Arch Linux / Manjaro
```bash
# Update package database
sudo pacman -Syu
# Install Node.js and npm
sudo pacman -S nodejs npm
# Verify installation
node -v
npm -v
````
---
## 2. Debian / Ubuntu
```bash
# Update package database and install curl
sudo apt update
sudo apt install -y curl
# Install nvm (Node Version Manager)
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.4/install.sh | bash
# Load nvm
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"
# Reload shell
source ~/.bashrc # Or source ~/.zshrc if using Zsh
# Install latest Node.js and npm
nvm install node
# Verify installation
node -v
npm -v
```
---
## 3. CentOS / RHEL
```bash
# Install curl if missing
sudo yum install -y curl
# Install nvm
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.4/install.sh | bash
# Load nvm
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"
# Reload shell
source ~/.bashrc # Or source ~/.zshrc if using Zsh
# Install latest Node.js and npm
nvm install node
# Verify installation
node -v
npm -v
```
---
## 4. Fedora
```bash
# Update system
sudo dnf update -y
# Install Node.js and npm from modules
sudo dnf module list nodejs # Check available versions
sudo dnf module enable nodejs:18 # Example: enable Node 18 LTS
sudo dnf install -y nodejs npm
# Verify installation
node -v
npm -v
```
---
## 5. openSUSE
```bash
# Refresh repositories
sudo zypper refresh
# Install Node.js and npm
sudo zypper install -y nodejs npm
# Verify installation
node -v
npm -v
```
---
## 6. Using nvm (Optional, Recommended)
`nvm` allows installing multiple Node.js versions and switching between them easily:
```bash
# Install nvm
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.4/install.sh | bash
# Load nvm
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"
# Reload shell
source ~/.bashrc # Or source ~/.zshrc if using Zsh
# Install latest Node.js and npm
nvm install node
# Or install latest LTS version
nvm install --lts
# Switch Node versions
nvm use node
# Verify installation
node -v
npm -v
```
package-lock.json
Generated
+30 -168
View File
@@ -454,6 +454,7 @@
"resolved": "https://registry.npmjs.org/@aws-sdk/client-dynamodb/-/client-dynamodb-3.1032.0.tgz",
"integrity": "sha512-kkXiZBNdWCQAg/8opqAu10TxzdpqMkcGrNAT2ScdfWhCpzYZ2pmSpP8W7BOlA32jYIWnYrEdb808UZsNWYBPAA==",
"license": "Apache-2.0",
"peer": true,
"dependencies": {
"@aws-crypto/sha256-browser": "5.2.0",
"@aws-crypto/sha256-js": "5.2.0",
@@ -1749,6 +1750,7 @@
}
],
"license": "MIT",
"peer": true,
"engines": {
"node": ">=20.19.0"
},
@@ -1797,6 +1799,7 @@
}
],
"license": "MIT",
"peer": true,
"engines": {
"node": ">=20.19.0"
}
@@ -1811,28 +1814,6 @@
"node": ">=10.0.0"
}
},
"node_modules/@emnapi/core": {
"version": "1.9.2",
"resolved": "https://registry.npmjs.org/@emnapi/core/-/core-1.9.2.tgz",
"integrity": "sha512-UC+ZhH3XtczQYfOlu3lNEkdW/p4dsJ1r/bP7H8+rhao3TTTMO1ATq/4DdIi23XuGoFY+Cz0JmCbdVl0hz9jZcA==",
"dev": true,
"license": "MIT",
"optional": true,
"dependencies": {
"@emnapi/wasi-threads": "1.2.1",
"tslib": "^2.4.0"
}
},
"node_modules/@emnapi/runtime": {
"version": "1.10.0",
"resolved": "https://registry.npmjs.org/@emnapi/runtime/-/runtime-1.10.0.tgz",
"integrity": "sha512-ewvYlk86xUoGI0zQRNq/mC+16R1QeDlKQy21Ki3oSYXNgLb45GV1P6A0M+/s6nyCuNDqe5VpaY84BzXGwVbwFA==",
"license": "MIT",
"optional": true,
"dependencies": {
"tslib": "^2.4.0"
}
},
"node_modules/@emnapi/wasi-threads": {
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/@emnapi/wasi-threads/-/wasi-threads-1.2.1.tgz",
@@ -2511,9 +2492,6 @@
"cpu": [
"arm"
],
"libc": [
"glibc"
],
"license": "LGPL-3.0-or-later",
"optional": true,
"os": [
@@ -2530,9 +2508,6 @@
"cpu": [
"arm64"
],
"libc": [
"glibc"
],
"license": "LGPL-3.0-or-later",
"optional": true,
"os": [
@@ -2549,9 +2524,6 @@
"cpu": [
"ppc64"
],
"libc": [
"glibc"
],
"license": "LGPL-3.0-or-later",
"optional": true,
"os": [
@@ -2568,9 +2540,6 @@
"cpu": [
"riscv64"
],
"libc": [
"glibc"
],
"license": "LGPL-3.0-or-later",
"optional": true,
"os": [
@@ -2587,9 +2556,6 @@
"cpu": [
"s390x"
],
"libc": [
"glibc"
],
"license": "LGPL-3.0-or-later",
"optional": true,
"os": [
@@ -2606,9 +2572,6 @@
"cpu": [
"x64"
],
"libc": [
"glibc"
],
"license": "LGPL-3.0-or-later",
"optional": true,
"os": [
@@ -2625,9 +2588,6 @@
"cpu": [
"arm64"
],
"libc": [
"musl"
],
"license": "LGPL-3.0-or-later",
"optional": true,
"os": [
@@ -2644,9 +2604,6 @@
"cpu": [
"x64"
],
"libc": [
"musl"
],
"license": "LGPL-3.0-or-later",
"optional": true,
"os": [
@@ -2663,9 +2620,6 @@
"cpu": [
"arm"
],
"libc": [
"glibc"
],
"license": "Apache-2.0",
"optional": true,
"os": [
@@ -2688,9 +2642,6 @@
"cpu": [
"arm64"
],
"libc": [
"glibc"
],
"license": "Apache-2.0",
"optional": true,
"os": [
@@ -2713,9 +2664,6 @@
"cpu": [
"ppc64"
],
"libc": [
"glibc"
],
"license": "Apache-2.0",
"optional": true,
"os": [
@@ -2738,9 +2686,6 @@
"cpu": [
"riscv64"
],
"libc": [
"glibc"
],
"license": "Apache-2.0",
"optional": true,
"os": [
@@ -2763,9 +2708,6 @@
"cpu": [
"s390x"
],
"libc": [
"glibc"
],
"license": "Apache-2.0",
"optional": true,
"os": [
@@ -2788,9 +2730,6 @@
"cpu": [
"x64"
],
"libc": [
"glibc"
],
"license": "Apache-2.0",
"optional": true,
"os": [
@@ -2813,9 +2752,6 @@
"cpu": [
"arm64"
],
"libc": [
"musl"
],
"license": "Apache-2.0",
"optional": true,
"os": [
@@ -2838,9 +2774,6 @@
"cpu": [
"x64"
],
"libc": [
"musl"
],
"license": "Apache-2.0",
"optional": true,
"os": [
@@ -3146,6 +3079,7 @@
"resolved": "https://registry.npmjs.org/@opentelemetry/api/-/api-1.9.1.tgz",
"integrity": "sha512-gLyJlPHPZYdAk1JENA9LeHejZe1Ti77/pTeFm/nMXmQH/HFZlcS/O2XJB+L8fkbrNSqhdtlvjBVjxwUYanNH5Q==",
"license": "Apache-2.0",
"peer": true,
"engines": {
"node": ">=8.0.0"
}
@@ -5697,9 +5631,6 @@
"arm64"
],
"dev": true,
"libc": [
"glibc"
],
"license": "MIT",
"optional": true,
"os": [
@@ -5717,9 +5648,6 @@
"arm64"
],
"dev": true,
"libc": [
"musl"
],
"license": "MIT",
"optional": true,
"os": [
@@ -5737,9 +5665,6 @@
"ppc64"
],
"dev": true,
"libc": [
"glibc"
],
"license": "MIT",
"optional": true,
"os": [
@@ -5757,9 +5682,6 @@
"s390x"
],
"dev": true,
"libc": [
"glibc"
],
"license": "MIT",
"optional": true,
"os": [
@@ -5777,9 +5699,6 @@
"x64"
],
"dev": true,
"libc": [
"glibc"
],
"license": "MIT",
"optional": true,
"os": [
@@ -5797,9 +5716,6 @@
"x64"
],
"dev": true,
"libc": [
"musl"
],
"license": "MIT",
"optional": true,
"os": [
@@ -6885,6 +6801,7 @@
"integrity": "sha512-FXx2pKgId/WyYo2jXw63kk7/+TY7u7AziEJxJAnSFzHlqTAS3Ync6SvgYAN/k4/PQpnnVuzoMuVnByKK2qp0ag==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"@types/estree": "*",
"@types/json-schema": "*"
@@ -7029,6 +6946,7 @@
"resolved": "https://registry.npmjs.org/@types/node/-/node-24.12.2.tgz",
"integrity": "sha512-A1sre26ke7HDIuY/M23nd9gfB+nrmhtYyMINbjI1zHJxYteKR6qSMX56FsmjMcDb3SMcjJg5BiRRgOCC/yBD0g==",
"license": "MIT",
"peer": true,
"dependencies": {
"undici-types": "~7.16.0"
}
@@ -7177,6 +7095,7 @@
"integrity": "sha512-/Zb/xaIDfxeJnvishjGdcR4jmr7S+bda8PKNhRGdljDM+elXhlvN0FyPSsMnLmJUrVG9aPO6dof80wjMawsASg==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"@typescript-eslint/scope-manager": "8.58.2",
"@typescript-eslint/types": "8.58.2",
@@ -7462,6 +7381,7 @@
"integrity": "sha512-x7FptB5oDruxNPDNY2+S8tCh0pcq7ymCe1gTHcsp733jYjrJl8V1gMUlVysuCD9Kz46Xz9t1akkv08dPcYDs1w==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"@bcoe/v8-coverage": "^1.0.2",
"@vitest/utils": "4.1.4",
@@ -7601,6 +7521,7 @@
"integrity": "sha512-EgFR7nlj5iTDYZYCvavjFokNYwr3c3ry0sFiCg+N7B233Nwp+NNx7eoF/XvMWDCKY71xXAG3kFkt97ZHBJVL8A==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"@vitest/utils": "4.1.4",
"fflate": "^0.8.2",
@@ -7938,6 +7859,7 @@
"resolved": "https://registry.npmjs.org/acorn/-/acorn-8.16.0.tgz",
"integrity": "sha512-UVJyE9MttOsBQIDKw1skb9nAwQuR5wuGD3+82K6JgJlm/Y+KI92oNsMNGZCYdDsVtRHSak0pcV5Dno5+4jh9sw==",
"license": "MIT",
"peer": true,
"bin": {
"acorn": "bin/acorn"
},
@@ -8651,6 +8573,7 @@
}
],
"license": "MIT",
"peer": true,
"dependencies": {
"baseline-browser-mapping": "^2.10.12",
"caniuse-lite": "^1.0.30001782",
@@ -8849,6 +8772,7 @@
"resolved": "https://registry.npmjs.org/chai/-/chai-4.5.0.tgz",
"integrity": "sha512-RITGBfijLkBddZvnn8jdqoTypxvqbOLYQkGGxXzeFjVHvudaPw0HNFD9x928/eUwYWd2dPCugVqspGALTZZQKw==",
"license": "MIT",
"peer": true,
"dependencies": {
"assertion-error": "^1.1.0",
"check-error": "^1.0.3",
@@ -9097,10 +9021,6 @@
"node": ">= 12"
}
},
"node_modules/comment-parser": {
"resolved": "tools/comment-parser",
"link": true
},
"node_modules/compressible": {
"version": "2.0.18",
"resolved": "https://registry.npmjs.org/compressible/-/compressible-2.0.18.tgz",
@@ -10235,6 +10155,7 @@
"integrity": "sha512-XoMjdBOwe/esVgEvLmNsD3IRHkm7fbKIUGvrleloJXUZgDHig2IPWNniv+GwjyJXzuNqVjlr5+4yVUZjycJwfQ==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"@eslint-community/eslint-utils": "^4.8.0",
"@eslint-community/regexpp": "^4.12.1",
@@ -10295,6 +10216,7 @@
"integrity": "sha512-82GZUjRS0p/jganf6q1rEO25VSoHH0hKPCTrgillPjdI/3bgBhAE1QzHrHTizjpRvy6pGAvKjDJtk2pF9NDq8w==",
"dev": true,
"license": "MIT",
"peer": true,
"bin": {
"eslint-config-prettier": "bin/cli.js"
},
@@ -10941,6 +10863,7 @@
"resolved": "https://registry.npmjs.org/fengari/-/fengari-0.1.5.tgz",
"integrity": "sha512-0DS4Nn4rV8qyFlQCpKK8brT61EUtswynrpfFTcgLErcilBIBskSMQ86fO2WVuybr14ywyKdRjv91FiRZwnEuvQ==",
"license": "MIT",
"peer": true,
"dependencies": {
"readline-sync": "^1.4.10",
"sprintf-js": "^1.1.3",
@@ -11023,10 +10946,6 @@
"integrity": "sha512-0Zt+s3L7Vf1biwWZ29aARiVYLx7iMGnEUl9x33fbB/j3jR81u/O2LbqK+Bm1CDSNDKVtJ/YjwY7TUd5SkeLQLw==",
"license": "MIT"
},
"node_modules/file-walker": {
"resolved": "tools/file-walker",
"link": true
},
"node_modules/fill-range": {
"version": "7.1.1",
"resolved": "https://registry.npmjs.org/fill-range/-/fill-range-7.1.1.tgz",
@@ -11466,10 +11385,6 @@
"node": ">= 0.4"
}
},
"node_modules/genwiki": {
"resolved": "tools/genwiki",
"link": true
},
"node_modules/get-caller-file": {
"version": "2.0.5",
"resolved": "https://registry.npmjs.org/get-caller-file/-/get-caller-file-2.0.5.tgz",
@@ -12371,6 +12286,7 @@
"resolved": "https://registry.npmjs.org/ioredis/-/ioredis-5.10.1.tgz",
"integrity": "sha512-HuEDBTI70aYdx1v6U97SbNx9F1+svQKBDo30o0b9fw055LMepzpOOd0Ccg9Q6tbqmBSJaMuY0fB7yw9/vjBYCA==",
"license": "MIT",
"peer": true,
"dependencies": {
"@ioredis/commands": "1.5.1",
"cluster-key-slot": "^1.1.0",
@@ -13839,9 +13755,6 @@
"arm64"
],
"dev": true,
"libc": [
"glibc"
],
"license": "MPL-2.0",
"optional": true,
"os": [
@@ -13863,9 +13776,6 @@
"arm64"
],
"dev": true,
"libc": [
"musl"
],
"license": "MPL-2.0",
"optional": true,
"os": [
@@ -13887,9 +13797,6 @@
"x64"
],
"dev": true,
"libc": [
"glibc"
],
"license": "MPL-2.0",
"optional": true,
"os": [
@@ -13911,9 +13818,6 @@
"x64"
],
"dev": true,
"libc": [
"musl"
],
"license": "MPL-2.0",
"optional": true,
"os": [
@@ -16216,6 +16120,7 @@
"integrity": "sha512-7igPTM53cGHMW8xWuVTydi2KO233VFiTNyF5hLJqpilHfmn8C8gPf+PS7dUT64YcXFbiMGZxS9pCSxL/Dxm/Jw==",
"dev": true,
"license": "MIT",
"peer": true,
"bin": {
"prettier": "bin/prettier.cjs"
},
@@ -17080,6 +16985,7 @@
"integrity": "sha512-PlXPeEWMXMZ7sPYOHqmDyCJzcfNrUr3fGNKtezX14ykXOEIvyK81d+qydx89KY5O71FKMPaQ2vBfBFI5NHR63A==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"fast-deep-equal": "^3.1.3",
"fast-uri": "^3.0.1",
@@ -17662,6 +17568,7 @@
"resolved": "https://registry.npmjs.org/socket.io-adapter/-/socket.io-adapter-2.5.6.tgz",
"integrity": "sha512-DkkO/dz7MGln0dHn5bmN3pPy+JmywNICWrJqVWiVOyvXjWQFIv9c2h24JrQLLFJ2aQVQf/Cvl1vblnd4r2apLQ==",
"license": "MIT",
"peer": true,
"dependencies": {
"debug": "~4.4.1",
"ws": "~8.18.3"
@@ -18632,6 +18539,7 @@
"integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==",
"devOptional": true,
"license": "Apache-2.0",
"peer": true,
"bin": {
"tsc": "bin/tsc",
"tsserver": "bin/tsserver"
@@ -18810,10 +18718,6 @@
"dev": true,
"license": "MIT"
},
"node_modules/useapi": {
"resolved": "src/useapi",
"link": true
},
"node_modules/util-deprecate": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/util-deprecate/-/util-deprecate-1.0.2.tgz",
@@ -18878,6 +18782,7 @@
"integrity": "sha512-t7g7GVRpMXjNpa67HaVWI/8BWtdVIQPCL2WoozXXA7LBGEFK4AkkKkHx2hAQf5x1GZSlcmEDPkVLSGahxnEEZw==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"lightningcss": "^1.32.0",
"picomatch": "^4.0.4",
@@ -19073,6 +18978,7 @@
"integrity": "sha512-tFuJqTxKb8AvfyqMfnavXdzfy3h3sWZRWwfluGbkeR7n0HUev+FmNgZ8SDrRBTVrVCjgH5cA21qGbCffMNtWvg==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"@vitest/expect": "4.1.4",
"@vitest/mocker": "4.1.4",
@@ -19204,6 +19110,7 @@
"integrity": "sha512-wGN3qcrBQIFmQ/c0AiOAQBvrZ5lmY8vbbMv4Mxfgzqd/B6+9pXtLo73WuS1dSGXM5QYY3hZnIbvx+K1xxe6FyA==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"@types/eslint-scope": "^3.7.7",
"@types/estree": "^1.0.8",
@@ -19252,6 +19159,7 @@
"integrity": "sha512-pIDJHIEI9LR0yxHXQ+Qh95k2EvXpWzZ5l+d+jIo+RdSm9MiHfzazIxwwni/p7+x4eJZuvG1AJwgC4TNQ7NRgsg==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"@discoveryjs/json-ext": "^0.5.0",
"@webpack-cli/configtest": "^2.1.1",
@@ -19707,6 +19615,7 @@
"integrity": "sha512-AvbaCLOO2Otw/lW5bmh9d/WEdcDFdQp2Z2ZUH3pX9U2ihyUY0nvLv7J6TrWowklRGPYbB/IuIMfYgxaCPg5Bpg==",
"dev": true,
"license": "ISC",
"peer": true,
"bin": {
"yaml": "bin.mjs"
},
@@ -19985,6 +19894,7 @@
"resolved": "https://registry.npmjs.org/zod/-/zod-4.3.6.tgz",
"integrity": "sha512-rftlrkhHZOcjDwkGlnUtZZkvaPHCsDATp4pGpuOOMDaTdDDXF91wuVDJoWoPsKX/3YPQ5fHuF3STjcYyKr+Qhg==",
"license": "MIT",
"peer": true,
"funding": {
"url": "https://github.com/sponsors/colinhacks"
}
@@ -20497,6 +20407,7 @@
},
"src/useapi": {
"version": "1.0.0",
"extraneous": true,
"license": "AGPL-3.0-only"
},
"src/worker": {
@@ -20511,69 +20422,20 @@
},
"tools/comment-parser": {
"version": "1.0.0",
"extraneous": true,
"license": "AGPL-3.0-only",
"devDependencies": {
"chai": "^5.1.1"
}
},
"tools/comment-parser/node_modules/assertion-error": {
"version": "2.0.1",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=12"
}
},
"tools/comment-parser/node_modules/chai": {
"version": "5.3.3",
"dev": true,
"license": "MIT",
"dependencies": {
"assertion-error": "^2.0.1",
"check-error": "^2.1.1",
"deep-eql": "^5.0.1",
"loupe": "^3.1.0",
"pathval": "^2.0.0"
},
"engines": {
"node": ">=18"
}
},
"tools/comment-parser/node_modules/check-error": {
"version": "2.1.3",
"dev": true,
"license": "MIT",
"engines": {
"node": ">= 16"
}
},
"tools/comment-parser/node_modules/deep-eql": {
"version": "5.0.2",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=6"
}
},
"tools/comment-parser/node_modules/loupe": {
"version": "3.2.1",
"dev": true,
"license": "MIT"
},
"tools/comment-parser/node_modules/pathval": {
"version": "2.0.1",
"dev": true,
"license": "MIT",
"engines": {
"node": ">= 14.16"
}
},
"tools/file-walker": {
"version": "1.0.0",
"extraneous": true,
"license": "AGPL-3.0-only"
},
"tools/genwiki": {
"version": "0.0.0",
"extraneous": true,
"license": "AGPL-3.0-only"
},
"tools/migrations-test": {
src/gui/src/lib/jquery-3.6.1/jquery-3.6.1.min.map
-1
View File
File diff suppressed because one or more lines are too long
src/gui/src/lib/socket.io/socket.io.js.map
-1
View File
File diff suppressed because one or more lines are too long
src/gui/src/lib/socket.io/socket.io.min.js.map
-1
View File
File diff suppressed because one or more lines are too long
src/puter-js/src/lib/socket.io/socket.io.esm.min.js.map
-1
View File
File diff suppressed because one or more lines are too long
src/puter-js/src/lib/socket.io/socket.io.js.map
-1
View File
File diff suppressed because one or more lines are too long
src/puter-js/src/lib/socket.io/socket.io.min.js.map
-1
View File
File diff suppressed because one or more lines are too long
src/puter-js/src/lib/socket.io/socket.io.msgpack.min.js.map
-1
View File
File diff suppressed because one or more lines are too long
src/puter-wisp/README.md
-102
View File
@@ -1,102 +0,0 @@
# Wisp Utilities
This is still a work in progress. Thes utilities use my own stream interface
to avoid browser/node compatibility issues and because I found it more
convenient. These streams can by used as async iterator objects just like
other conventional implementations. Currently there is no logic for closing
streams or knowing if a stream has been closed, but this is planned.
## Classes and Factory Functions
### WispPacket (class)
Wraps a Uint8Array containing a Wisp packet. `data` should be a Uint8Array
containing only the Wisp frame, starting at the Packet Type and ending at
the last byte of the payload (inclusive).
```javascript
const packet = new WispPacket({
data: new Uint8Array(...),
direction: WispPacket.SEND, // or RECV
// `extra` is optional, for debugging
extra: { some: 'value', },
});
packet.type; // ex: WispPacket.CONTINUE
```
#### Methods
- `describe()` - outputs a summary string
```javascript
packet.describe();
// ex: "INFO v2.0 f000000000"
```
- `toVirtioFrame` - prepends the size of the Wisp frame (u32LE)
- `log()` - prints a collapsed console group
- `reflect()` - returns a reflected version of the packet (flips `SEND` and `RECV`)
### NewCallbackByteStream (function)
Returns a stream for values that get passed through a callback interface.
The stream object (an async iterator object) has a property called
`listener` which can be passed as a listener or called directly. This
listener expects only one argument which is the data to pass through the
stream (typically a value of type `Uint8Array`).
```javascript
const byteStream = NewCallbackByteStream();
emulator.add_listener('virtio-console0-output-bytes',
byteStream.listener);
```
### NewVirtioFrameStream (function)
Takes in a byte stream (stream of `Uint8Array`) and assumes that this byte
stream contains integers (u32LE) describing the length (in bytes) of data,
followed by the data. Returns a stream which outputs a complete chunk of
data (as per the specified length) as each value, excluding the bytes that
describe the length.
```javascript
const virtioStream = NewVirtioFrameStream(byteStream);
```
### NewWispPacketStream (function)
Takes in a stream of `Uint8Array`s, each containing a complete Wisp packet,
and outputs a stream of instances of **WispPacket**
## Example Use with v86
```javascript
const emulator = new V86(...);
// Get a byte stream for /dev/hvc0
const byteStream = NewCallbackByteStream();
emulator.add_listener('virtio-console0-output-bytes',
byteStream.listener);
// Get a stream of frames with prepended byte lengths
// (for example, `twisp` uses this format)
const virtioStream = NewVirtioFrameStream(byteStream);
// Get a stream of WispPacket objects
const wispStream = NewWispPacketStream(virtioStream);
// Async iterator
(async () => {
for ( const packet of wispStream ) {
console.log('Wisp packet!', packet.describe());
// Let's send back a reflected packet for INFO!
if ( packet.type === WispPacket.INFO ) {
emulator.bus.send(
'virtio-console0-input-bytes',
packet.toVirtioFrame(),
);
}
}
})();
```
src/puter-wisp/basic.html
-412
View File
@@ -1,412 +0,0 @@
<!doctype html>
<title>Basic Emulator</title><!-- not BASIC! -->
<style>
div {
font-size: 12px;
line-height: 16px;
}
BODY {
background-color: #111;
display: flex;
justify-content: center;
align-items: center;
height: 100vh;
overflow: hidden;
}
</style>
<script src="../build/libv86.js"></script>
<script>
"use strict";
// Libs
// SO: 40031688
function buf2hex(buffer) { // buffer is an ArrayBuffer
return [...new Uint8Array(buffer)]
.map(x => x.toString(16).padStart(2, '0'))
.join('');
}
class ATStream {
constructor ({ delegate, acc, transform, observe }) {
this.delegate = delegate;
if ( acc ) this.acc = acc;
if ( transform ) this.transform = transform;
if ( observe ) this.observe = observe;
this.state = {};
this.carry = [];
}
[Symbol.asyncIterator]() { return this; }
async next_value_ () {
if ( this.carry.length > 0 ) {
console.log('got from carry!', this.carry);
return {
value: this.carry.shift(),
done: false,
};
}
return await this.delegate.next();
}
async acc ({ value }) {
return value;
}
async next_ () {
for (;;) {
const ret = await this.next_value_();
if ( ret.done ) return ret;
const v = await this.acc({
state: this.state,
value: ret.value,
carry: v => this.carry.push(v),
});
if ( this.carry.length >= 0 && v === undefined ) {
throw new Error(`no value, but carry value exists`);
}
if ( v === undefined ) continue;
// We have a value, clear the state!
this.state = {};
if ( this.transform ) {
const new_value = await this.transform(
{ value: ret.value });
return { ...ret, value: new_value };
}
return { ...ret, value: v };
}
}
async next () {
const ret = await this.next_();
if ( this.observe && !ret.done ) {
this.observe(ret);
}
return ret;
}
async enqueue_ (v) {
this.queue.push(v);
}
}
const NewCallbackByteStream = () => {
let listener;
let queue = [];
const NOOP = () => {};
let signal = NOOP;
(async () => {
for (;;) {
const v = await new Promise((rslv, rjct) => {
listener = rslv;
});
queue.push(v);
signal();
}
})();
const stream = {
[Symbol.asyncIterator](){
return this;
},
async next () {
if ( queue.length > 0 ) {
return {
value: queue.shift(),
done: false,
};
}
await new Promise(rslv => {
signal = rslv;
});
signal = NOOP;
const v = queue.shift();
return { value: v, done: false };
}
};
stream.listener = data => {
listener(data);
};
return stream;
}
// Tiny inline little-endian integer library
const get_int = (n_bytes, array8, signed=false) => {
return (v => signed ? v : v >>> 0)(
array8.slice(0,n_bytes).reduce((v,e,i)=>v|=e<<8*i,0));
}
const to_int = (n_bytes, num) => {
return (new Uint8Array()).map((_,i)=>(num>>8*i)&0xFF);
}
const NewVirtioFrameStream = byteStream => {
return new ATStream({
delegate: byteStream,
async acc ({ value, carry }) {
if ( ! this.state.buffer ) {
const size = get_int(4, value);
// 512MiB limit in case of attempted abuse or a bug
// (assuming this won't happen under normal conditions)
if ( size > 512*(1024**2) ) {
throw new Error(`Way too much data! (${size} bytes)`);
}
value = value.slice(4);
this.state.buffer = new Uint8Array(size);
this.state.index = 0;
}
const needed = this.state.buffer.length - this.state.index;
if ( value.length > needed ) {
const remaining = value.slice(needed);
console.log('we got more bytes than we needed',
needed,
remaining,
value.length,
this.state.buffer.length,
this.state.index,
);
carry(remaining);
}
const amount = Math.min(value.length, needed);
const added = value.slice(0, amount);
this.state.buffer.set(added, this.state.index);
this.state.index += amount;
if ( this.state.index > this.state.buffer.length ) {
throw new Error('WUT');
}
if ( this.state.index == this.state.buffer.length ) {
return this.state.buffer;
}
}
});
};
const wisp_types = [
{
id: 3,
label: 'CONTINUE',
describe: ({ payload }) => {
return `buffer: ${get_int(4, payload)}B`;
},
getAttributes ({ payload }) {
return {
buffer_size: get_int(4, payload),
};
}
},
{
id: 5,
label: 'INFO',
describe: ({ payload }) => {
return `v${payload[0]}.${payload[1]} ` +
buf2hex(payload.slice(2));
},
getAttributes ({ payload }) {
return {
version_major: payload[0],
version_minor: payload[1],
extensions: payload.slice(2),
}
}
},
];
class WispPacket {
static SEND = Symbol('SEND');
static RECV = Symbol('RECV');
constructor ({ data, direction, extra }) {
this.direction = direction;
this.data_ = data;
this.extra = extra ?? {};
this.types_ = {
1: { label: 'CONNECT' },
2: { label: 'DATA' },
4: { label: 'CLOSE' },
};
for ( const item of wisp_types ) {
this.types_[item.id] = item;
}
}
get type () {
const i_ = this.data_[0];
return this.types_[i_];
}
get attributes () {
if ( ! this.type.getAttributes ) return {};
const attrs = {};
Object.assign(attrs, this.type.getAttributes({
payload: this.data_.slice(5),
}));
Object.assign(attrs, this.extra);
return attrs;
}
toVirtioFrame () {
const arry = new Uint8Array(this.data_.length + 4);
arry.set(to_int(4, this.data_.length), 0);
arry.set(this.data_, 4);
return arry;
}
describe () {
return this.type.label + '(' +
(this.type.describe?.({
payload: this.data_.slice(5),
}) ?? '?') + ')';
}
log () {
const arrow =
this.direction === this.constructor.SEND ? '->' :
this.direction === this.constructor.RECV ? '<-' :
'<>' ;
console.groupCollapsed(`WISP ${arrow} ${this.describe()}`);
const attrs = this.attributes;
for ( const k in attrs ) {
console.log(k, attrs[k]);
}
console.groupEnd();
}
reflect () {
const reflected = new WispPacket({
data: this.data_,
direction:
this.direction === this.constructor.SEND ?
this.constructor.RECV :
this.direction === this.constructor.RECV ?
this.constructor.SEND :
undefined,
extra: {
reflectedFrom: this,
}
});
return reflected;
}
}
for ( const item of wisp_types ) {
WispPacket[item.label] = item;
}
const NewWispPacketStream = frameStream => {
return new ATStream({
delegate: frameStream,
transform ({ value }) {
return new WispPacket({
data: value,
direction: WispPacket.RECV,
});
},
observe ({ value }) {
value.log();
}
});
}
class WispClient {
constructor ({
packetStream,
sendFn,
}) {
this.packetStream = packetStream;
this.sendFn = sendFn;
}
send (packet) {
packet.log();
this.sendFn(packet);
}
}
window.onload = async function()
{
const resp = await fetch(
'./image/build/x86images/rootfs.bin'
);
const arrayBuffer = await resp.arrayBuffer();
var emulator = window.emulator = new V86({
wasm_path: "../build/v86.wasm",
memory_size: 512 * 1024 * 1024,
vga_memory_size: 2 * 1024 * 1024,
screen_container: document.getElementById("screen_container"),
bios: {
url: "../bios/seabios.bin",
},
vga_bios: {
url: "../bios/vgabios.bin",
},
initrd: {
url: './image/build/x86images/boot/initramfs-lts',
},
bzimage: {
url: './image/build/x86images/boot/vmlinuz-lts',
async: false
},
cmdline: 'rw root=/dev/sda init=/sbin/init rootfstype=ext4',
// cmdline: 'rw root=/dev/sda init=/bin/bash rootfstype=ext4',
// cmdline: "rw init=/sbin/init root=/dev/sda rootfstype=ext4",
// cmdline: "rw init=/sbin/init root=/dev/sda rootfstype=ext4 random.trust_cpu=on 8250.nr_uarts=10 spectre_v2=off pti=off mitigations=off",
// cdrom: {
// // url: "../images/al32-2024.07.10.iso",
// url: "./image/build/x86images/rootfs.bin",
// },
hda: {
buffer: arrayBuffer,
// url: './image/build/x86images/rootfs.bin',
async: true,
// size: 1073741824,
// size: 805306368,
},
// bzimage_initrd_from_filesystem: true,
autostart: true,
network_relay_url: "wisp://127.0.0.1:3000",
virtio_console: true,
});
const decoder = new TextDecoder();
const byteStream = NewCallbackByteStream();
emulator.add_listener('virtio-console0-output-bytes',
byteStream.listener);
const virtioStream = NewVirtioFrameStream(byteStream);
const wispStream = NewWispPacketStream(virtioStream);
class PTYManager {
constructor ({ client }) {
this.client = client;
}
init () {
this.run_();
}
async run_ () {
const handlers_ = {
[WispPacket.INFO.id]: ({ packet }) => {
// console.log('guess we doing info packets now', packet);
this.client.send(packet.reflect());
}
};
for await ( const packet of this.client.packetStream ) {
// console.log('what we got here?',
// packet.type,
// packet,
// );
handlers_[packet.type.id]?.({ packet });
}
}
}
const ptyMgr = new PTYManager({
client: new WispClient({
packetStream: wispStream,
sendFn: packet => {
emulator.bus.send(
"virtio-console0-input-bytes",
packet.toVirtioFrame(),
);
}
})
});
ptyMgr.init();
}
</script>
<!-- A minimal structure for the ScreenAdapter defined in browser/screen.js -->
<div id="screen_container">
<div style="white-space: pre; font: 14px monospace; line-height: 14px"></div>
<canvas style="display: none"></canvas>
</div>
src/puter-wisp/devlog/unit_test_usefulness/a.js
-317
View File
@@ -1,317 +0,0 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
const lib = {};
// SO: 40031688
lib.buf2hex = (buffer) => { // buffer is an ArrayBuffer
return [...new Uint8Array(buffer)]
.map(x => x.toString(16).padStart(2, '0'))
.join('');
};
// Tiny inline little-endian integer library
lib.get_int = (n_bytes, array8, signed = false) => {
return (v => signed ? v : v >>> 0)(
array8.slice(0, n_bytes).reduce((v, e, i) => v |= e << 8 * i, 0));
};
lib.to_int = (n_bytes, num) => {
return (new Uint8Array()).map((_, i) => (num >> 8 * i) & 0xFF);
};
class ATStream {
constructor ({ delegate, acc, transform, observe }) {
this.delegate = delegate;
if ( acc ) this.acc = acc;
if ( transform ) this.transform = transform;
if ( observe ) this.observe = observe;
this.state = {};
this.carry = [];
}
[Symbol.asyncIterator] () {
return this;
}
async next_value_ () {
if ( this.carry.length > 0 ) {
console.log('got from carry!', this.carry);
return {
value: this.carry.shift(),
done: false,
};
}
return await this.delegate.next();
}
async acc ({ value }) {
return value;
}
async next_ () {
for ( ;; ) {
const ret = await this.next_value_();
if ( ret.done ) return ret;
const v = await this.acc({
state: this.state,
value: ret.value,
carry: v => this.carry.push(v),
});
if ( this.carry.length >= 0 && v === undefined ) {
throw new Error('no value, but carry value exists');
}
if ( v === undefined ) continue;
// We have a value, clear the state!
this.state = {};
if ( this.transform ) {
const new_value = await this.transform({ value: ret.value });
return { ...ret, value: new_value };
}
return { ...ret, value: v };
}
}
async next () {
const ret = await this.next_();
if ( this.observe && !ret.done ) {
this.observe(ret);
}
return ret;
}
async enqueue_ (v) {
this.queue.push(v);
}
}
const NewCallbackByteStream = () => {
let listener;
let queue = [];
const NOOP = () => {
};
let signal = NOOP;
(async () => {
for ( ;; ) {
const v = await new Promise((rslv, rjct) => {
listener = rslv;
});
queue.push(v);
signal();
}
})();
const stream = {
[Symbol.asyncIterator] () {
return this;
},
async next () {
if ( queue.length > 0 ) {
return {
value: queue.shift(),
done: false,
};
}
await new Promise(rslv => {
signal = rslv;
});
signal = NOOP;
const v = queue.shift();
return { value: v, done: false };
},
};
stream.listener = data => {
listener(data);
};
return stream;
};
const NewVirtioFrameStream = byteStream => {
return new ATStream({
delegate: byteStream,
async acc ({ value, carry }) {
if ( ! this.state.buffer ) {
const size = lib.get_int(4, value);
// 512MiB limit in case of attempted abuse or a bug
// (assuming this won't happen under normal conditions)
if ( size > 512 * (1024 ** 2) ) {
throw new Error(`Way too much data! (${size} bytes)`);
}
value = value.slice(4);
this.state.buffer = new Uint8Array(size);
this.state.index = 0;
}
const needed = this.state.buffer.length - this.state.index;
if ( value.length > needed ) {
const remaining = value.slice(needed);
console.log('we got more bytes than we needed',
needed,
remaining,
value.length,
this.state.buffer.length,
this.state.index);
carry(remaining);
}
const amount = Math.min(value.length, needed);
const added = value.slice(0, amount);
this.state.buffer.set(added, this.state.index);
this.state.index += amount;
if ( this.state.index > this.state.buffer.length ) {
throw new Error('WUT');
}
if ( this.state.index == this.state.buffer.length ) {
return this.state.buffer;
}
},
});
};
const wisp_types = [
{
id: 3,
label: 'CONTINUE',
describe: ({ payload }) => {
return `buffer: ${lib.get_int(4, payload)}B`;
},
getAttributes ({ payload }) {
return {
buffer_size: lib.get_int(4, payload),
};
},
},
{
id: 5,
label: 'INFO',
describe: ({ payload }) => {
return `v${payload[0]}.${payload[1]} ${
lib.buf2hex(payload.slice(2))}`;
},
getAttributes ({ payload }) {
return {
version_major: payload[0],
version_minor: payload[1],
extensions: payload.slice(2),
};
},
},
];
class WispPacket {
static SEND = Symbol('SEND');
static RECV = Symbol('RECV');
constructor ({ data, direction, extra }) {
this.direction = direction;
this.data_ = data;
this.extra = extra ?? {};
this.types_ = {
1: { label: 'CONNECT' },
2: { label: 'DATA' },
4: { label: 'CLOSE' },
};
for ( const item of wisp_types ) {
this.types_[item.id] = item;
}
}
get type () {
const i_ = this.data_[0];
return this.types_[i_];
}
get attributes () {
if ( ! this.type.getAttributes ) return {};
const attrs = {};
Object.assign(attrs, this.type.getAttributes({
payload: this.data_.slice(5),
}));
Object.assign(attrs, this.extra);
return attrs;
}
toVirtioFrame () {
const arry = new Uint8Array(this.data_.length + 4);
arry.set(lib.to_int(4, this.data_.length), 0);
arry.set(this.data_, 4);
return arry;
}
describe () {
return `${this.type.label }(${
this.type.describe?.({
payload: this.data_.slice(5),
}) ?? '?' })`;
}
log () {
const arrow =
this.direction === this.constructor.SEND ? '->' :
this.direction === this.constructor.RECV ? '<-' :
'<>' ;
console.groupCollapsed(`WISP ${arrow} ${this.describe()}`);
const attrs = this.attributes;
for ( const k in attrs ) {
console.log(k, attrs[k]);
}
console.groupEnd();
}
reflect () {
const reflected = new WispPacket({
data: this.data_,
direction:
this.direction === this.constructor.SEND ?
this.constructor.RECV :
this.direction === this.constructor.RECV ?
this.constructor.SEND :
undefined,
extra: {
reflectedFrom: this,
},
});
return reflected;
}
}
for ( const item of wisp_types ) {
WispPacket[item.label] = item;
}
const NewWispPacketStream = frameStream => {
return new ATStream({
delegate: frameStream,
transform ({ value }) {
return new WispPacket({
data: value,
direction: WispPacket.RECV,
});
},
observe ({ value }) {
value.log();
},
});
};
class WispClient {
constructor ({
packetStream,
sendFn,
}) {
this.packetStream = packetStream;
this.sendFn = sendFn;
}
send (packet) {
packet.log();
this.sendFn(packet);
}
}
module.exports = {
NewVirtioFrameStream,
NewWispPacketStream,
WispPacket,
};
src/puter-wisp/devlog/unit_test_usefulness/b.js
-327
View File
@@ -1,327 +0,0 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
const lib = {};
// SO: 40031688
lib.buf2hex = (buffer) => { // buffer is an ArrayBuffer
return [...new Uint8Array(buffer)]
.map(x => x.toString(16).padStart(2, '0'))
.join('');
};
// Tiny inline little-endian integer library
lib.get_int = (n_bytes, array8, signed = false) => {
return (v => signed ? v : v >>> 0)(
array8.slice(0, n_bytes).reduce((v, e, i) => v |= e << 8 * i, 0));
};
lib.to_int = (n_bytes, num) => {
return (new Uint8Array()).map((_, i) => (num >> 8 * i) & 0xFF);
};
class ATStream {
constructor ({ delegate, acc, transform, observe }) {
this.delegate = delegate;
if ( acc ) this.acc = acc;
if ( transform ) this.transform = transform;
if ( observe ) this.observe = observe;
this.state = {};
this.carry = [];
}
[Symbol.asyncIterator] () {
return this;
}
async next_value_ () {
if ( this.carry.length > 0 ) {
return {
value: this.carry.shift(),
done: false,
};
}
return await this.delegate.next();
}
async acc ({ value }) {
return value;
}
async next_ () {
for ( ;; ) {
const ret = await this.next_value_();
if ( ret.done ) return ret;
const v = await this.acc({
state: this.state,
value: ret.value,
carry: v => this.carry.push(v),
});
if ( this.carry.length > 0 && v === undefined ) {
throw new Error('no value, but carry value exists');
}
if ( v === undefined ) continue;
// We have a value, clear the state!
this.state = {};
if ( this.transform ) {
const new_value = await this.transform({ value: ret.value });
return { ...ret, value: new_value };
}
return { ...ret, value: v };
}
}
async next () {
const ret = await this.next_();
if ( this.observe && !ret.done ) {
this.observe(ret);
}
return ret;
}
async enqueue_ (v) {
this.queue.push(v);
}
}
const NewCallbackByteStream = () => {
let listener;
let queue = [];
const NOOP = () => {
};
let signal = NOOP;
(async () => {
for ( ;; ) {
const v = await new Promise((rslv, rjct) => {
listener = rslv;
});
queue.push(v);
signal();
}
})();
const stream = {
[Symbol.asyncIterator] () {
return this;
},
async next () {
if ( queue.length > 0 ) {
return {
value: queue.shift(),
done: false,
};
}
await new Promise(rslv => {
signal = rslv;
});
signal = NOOP;
const v = queue.shift();
return { value: v, done: false };
},
};
stream.listener = data => {
listener(data);
};
return stream;
};
const NewVirtioFrameStream = byteStream => {
return new ATStream({
delegate: byteStream,
async acc ({ value, carry }) {
if ( ! this.state.buffer ) {
if ( this.state.hold ) {
const old_val = value;
let size = this.state.hold.length + value.length;
value = new Uint8Array(size);
value.set(this.state.hold, 0);
value.set(old_val, this.state.hold.length);
}
if ( value.length < 4 ) {
this.state.hold = value;
return undefined;
}
const size = lib.get_int(4, value);
// 512MiB limit in case of attempted abuse or a bug
// (assuming this won't happen under normal conditions)
if ( size > 512 * (1024 ** 2) ) {
throw new Error(`Way too much data! (${size} bytes)`);
}
value = value.slice(4);
this.state.buffer = new Uint8Array(size);
this.state.index = 0;
}
const needed = this.state.buffer.length - this.state.index;
if ( value.length > needed ) {
const remaining = value.slice(needed);
console.log('we got more bytes than we needed',
needed,
remaining,
value.length,
this.state.buffer.length,
this.state.index);
carry(remaining);
}
const amount = Math.min(value.length, needed);
const added = value.slice(0, amount);
this.state.buffer.set(added, this.state.index);
this.state.index += amount;
if ( this.state.index > this.state.buffer.length ) {
throw new Error('WUT');
}
if ( this.state.index == this.state.buffer.length ) {
return this.state.buffer;
}
},
});
};
const wisp_types = [
{
id: 3,
label: 'CONTINUE',
describe: ({ payload }) => {
return `buffer: ${lib.get_int(4, payload)}B`;
},
getAttributes ({ payload }) {
return {
buffer_size: lib.get_int(4, payload),
};
},
},
{
id: 5,
label: 'INFO',
describe: ({ payload }) => {
return `v${payload[0]}.${payload[1]} ${
lib.buf2hex(payload.slice(2))}`;
},
getAttributes ({ payload }) {
return {
version_major: payload[0],
version_minor: payload[1],
extensions: payload.slice(2),
};
},
},
];
class WispPacket {
static SEND = Symbol('SEND');
static RECV = Symbol('RECV');
constructor ({ data, direction, extra }) {
this.direction = direction;
this.data_ = data;
this.extra = extra ?? {};
this.types_ = {
1: { label: 'CONNECT' },
2: { label: 'DATA' },
4: { label: 'CLOSE' },
};
for ( const item of wisp_types ) {
this.types_[item.id] = item;
}
}
get type () {
const i_ = this.data_[0];
return this.types_[i_];
}
get attributes () {
if ( ! this.type.getAttributes ) return {};
const attrs = {};
Object.assign(attrs, this.type.getAttributes({
payload: this.data_.slice(5),
}));
Object.assign(attrs, this.extra);
return attrs;
}
toVirtioFrame () {
const arry = new Uint8Array(this.data_.length + 4);
arry.set(lib.to_int(4, this.data_.length), 0);
arry.set(this.data_, 4);
return arry;
}
describe () {
return `${this.type.label }(${
this.type.describe?.({
payload: this.data_.slice(5),
}) ?? '?' })`;
}
log () {
const arrow =
this.direction === this.constructor.SEND ? '->' :
this.direction === this.constructor.RECV ? '<-' :
'<>' ;
console.groupCollapsed(`WISP ${arrow} ${this.describe()}`);
const attrs = this.attributes;
for ( const k in attrs ) {
console.log(k, attrs[k]);
}
console.groupEnd();
}
reflect () {
const reflected = new WispPacket({
data: this.data_,
direction:
this.direction === this.constructor.SEND ?
this.constructor.RECV :
this.direction === this.constructor.RECV ?
this.constructor.SEND :
undefined,
extra: {
reflectedFrom: this,
},
});
return reflected;
}
}
for ( const item of wisp_types ) {
WispPacket[item.label] = item;
}
const NewWispPacketStream = frameStream => {
return new ATStream({
delegate: frameStream,
transform ({ value }) {
return new WispPacket({
data: value,
direction: WispPacket.RECV,
});
},
observe ({ value }) {
value.log();
},
});
};
class WispClient {
constructor ({
packetStream,
sendFn,
}) {
this.packetStream = packetStream;
this.sendFn = sendFn;
}
send (packet) {
packet.log();
this.sendFn(packet);
}
}
module.exports = {
NewVirtioFrameStream,
NewWispPacketStream,
WispPacket,
};
src/puter-wisp/devlog/unit_test_usefulness/test_a_b.diff
-18
View File
@@ -1,18 +0,0 @@
31d30
< console.log('got from carry!', this.carry);
51c50
< if ( this.carry.length >= 0 && v === undefined ) {
---
> if ( this.carry.length > 0 && v === undefined ) {
120a120,130
> if ( this.state.hold ) {
> const old_val = value;
> let size = this.state.hold.length + value.length;
> value = new Uint8Array(size);
> value.set(this.state.hold, 0);
> value.set(old_val, this.state.hold.length);
> }
> if ( value.length < 4 ) {
> this.state.hold = value;
> return undefined;
> }
src/puter-wisp/package.json
-15
View File
@@ -1,15 +0,0 @@
{
"name": "@heyputer/puter-wisp",
"version": "1.0.0",
"main": "exports.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"keywords": [],
"author": "",
"license": "AGPL-3.0-only",
"directories": {
"test": "test"
},
"description": ""
}
src/puter-wisp/src/exports.js
-431
View File
@@ -1,431 +0,0 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
const lib = {};
// SO: 40031688
lib.buf2hex = (buffer) => { // buffer is an ArrayBuffer
return [...new Uint8Array(buffer)]
.map(x => x.toString(16).padStart(2, '0'))
.join('');
};
// Tiny inline little-endian integer library
lib.get_int = (n_bytes, array8, signed = false) => {
return (v => signed ? v : v >>> 0)(
array8.slice(0, n_bytes).reduce((v, e, i) => v |= e << 8 * i, 0));
};
lib.to_int = (n_bytes, num) => {
return (new Uint8Array(n_bytes)).map((_, i) => (num >> 8 * i) & 0xFF);
};
// Accumulator and/or Transformer (and/or Observer) Stream
// The Swiss Army Knife* of Streams!
// (* this code is not affiliated with the Swiss Army Knife corporation)
class ATStream {
constructor ({ delegate, acc, transform, observe }) {
this.delegate = delegate;
if ( acc ) this.acc = acc;
if ( transform ) this.transform = transform;
if ( observe ) this.observe = observe;
this.state = {};
this.carry = [];
}
[Symbol.asyncIterator] () {
return this;
}
async next_value_ () {
if ( this.carry.length > 0 ) {
return {
value: this.carry.shift(),
done: false,
};
}
return await this.delegate.next();
}
async acc ({ value }) {
return value;
}
async next_ () {
for ( ;; ) {
const ret = await this.next_value_();
if ( ret.done ) return ret;
const v = await this.acc({
state: this.state,
value: ret.value,
carry: v => this.carry.push(v),
});
if ( this.carry.length > 0 && v === undefined ) {
throw new Error('no value, but carry value exists');
}
if ( v === undefined ) continue;
// We have a value, clear the state!
this.state = {};
if ( this.transform ) {
const new_value = await this.transform({ value: ret.value });
return { ...ret, value: new_value };
}
return { ...ret, value: v };
}
}
async next () {
const ret = await this.next_();
if ( this.observe && !ret.done ) {
this.observe(ret);
}
return ret;
}
async enqueue_ (v) {
this.queue.push(v);
}
}
const NewCallbackByteStream = () => {
let queue = [];
const NOOP = () => {
};
let signal = NOOP;
const stream = {
[Symbol.asyncIterator] () {
return this;
},
async next () {
if ( queue.length > 0 ) {
return {
value: queue.shift(),
done: false,
};
}
await new Promise(rslv => {
signal = rslv;
});
signal = NOOP;
const v = queue.shift();
return { value: v, done: false };
},
};
stream.listener = data => {
queue.push(data);
signal();
};
return stream;
};
const NewVirtioFrameStream = byteStream => {
return new ATStream({
delegate: byteStream,
async acc ({ value, carry }) {
if ( ! this.state.buffer ) {
if ( this.state.hold ) {
const old_val = value;
let size = this.state.hold.length + value.length;
value = new Uint8Array(size);
value.set(this.state.hold, 0);
value.set(old_val, this.state.hold.length);
}
if ( value.length < 4 ) {
this.state.hold = value;
return undefined;
}
const size = lib.get_int(4, value);
// 512MiB limit in case of attempted abuse or a bug
// (assuming this won't happen under normal conditions)
if ( size > 512 * (1024 ** 2) ) {
throw new Error(`Way too much data! (${size} bytes)`);
}
value = value.slice(4);
this.state.buffer = new Uint8Array(size);
this.state.index = 0;
}
const needed = this.state.buffer.length - this.state.index;
if ( value.length > needed ) {
const remaining = value.slice(needed);
console.log('we got more bytes than we needed',
needed,
remaining,
value.length,
this.state.buffer.length,
this.state.index);
carry(remaining);
}
const amount = Math.min(value.length, needed);
const added = value.slice(0, amount);
this.state.buffer.set(added, this.state.index);
this.state.index += amount;
if ( this.state.index > this.state.buffer.length ) {
throw new Error('WUT');
}
if ( this.state.index == this.state.buffer.length ) {
return this.state.buffer;
}
},
});
};
const wisp_types = [
{
id: 3,
label: 'CONTINUE',
describe: ({ payload }) => {
return `buffer: ${lib.get_int(4, payload)}B`;
},
getAttributes ({ payload }) {
return {
buffer_size: lib.get_int(4, payload),
};
},
},
{
id: 1,
label: 'CONNECT',
describe: ({ attributes }) => {
return `${
attributes.type === 1 ? 'TCP' :
attributes.type === 2 ? 'UDP' :
attributes.type === 3 ? 'PTY' :
'UNKNOWN'
} ${attributes.host}:${attributes.port}`;
},
getAttributes: ({ payload }) => {
const type = payload[0];
const port = lib.get_int(2, payload.slice(1));
const host = new TextDecoder().decode(payload.slice(3));
return {
type, port, host,
};
},
},
{
id: 5,
label: 'INFO',
describe: ({ payload }) => {
return `v${payload[0]}.${payload[1]} ${
lib.buf2hex(payload.slice(2))}`;
},
getAttributes ({ payload }) {
return {
version_major: payload[0],
version_minor: payload[1],
extensions: payload.slice(2),
};
},
},
{
id: 2,
label: 'DATA',
describe: ({ attributes }) => {
return `${attributes.length}B`;
},
getAttributes ({ payload }) {
return {
length: payload.length,
contents: payload,
utf8: new TextDecoder().decode(payload),
};
},
},
{
id: 4,
label: 'CLOSE',
describe: ({ attributes }) => {
return `reason: ${attributes.code}`;
},
getAttributes ({ payload }) {
return {
code: payload[0],
};
},
},
{
// TODO: extension types should not be hardcoded here
id: 0xf0,
label: 'RESIZE',
describe: ({ attributes }) => {
return `${attributes.cols}x${attributes.rows}`;
},
getAttributes ({ payload }) {
return {
rows: lib.get_int(2, payload),
cols: lib.get_int(2, payload.slice(2)),
};
},
},
];
class WispPacket {
static SEND = Symbol('SEND');
static RECV = Symbol('RECV');
constructor ({ data, direction, extra }) {
this.direction = direction;
this.data_ = data;
this.extra = extra ?? {};
this.types_ = {
4: { label: 'CLOSE' },
};
for ( const item of wisp_types ) {
this.types_[item.id] = item;
}
}
get type () {
const i_ = this.data_[0];
return this.types_[i_];
}
get attributes () {
if ( ! this.type.getAttributes ) return {};
const attrs = {
streamId: this.streamId,
};
Object.assign(attrs, this.type.getAttributes({
payload: this.data_.slice(5),
}));
Object.assign(attrs, this.extra);
return attrs;
}
get payload () {
return this.data_.slice(5);
}
get streamId () {
return lib.get_int(4, this.data_.slice(1));
}
toVirtioFrame () {
console.log('WISP packet to virtio frame',
this.data_,
this.data_.length,
lib.to_int(4, this.data_.length));
const arry = new Uint8Array(this.data_.length + 4);
arry.set(lib.to_int(4, this.data_.length), 0);
arry.set(this.data_, 4);
return arry;
}
describe () {
return `${this.type.label }(${
this.type.describe?.({
attributes: this.attributes,
payload: this.data_.slice(5),
}) ?? '?' })`;
}
log () {
const arrow =
this.direction === this.constructor.SEND ? '->' :
this.direction === this.constructor.RECV ? '<-' :
'<>' ;
console.groupCollapsed(`WISP ${arrow} ${this.describe()}`);
const attrs = this.attributes;
for ( const k in attrs ) {
console.log(k, attrs[k]);
}
console.groupEnd();
}
reflect () {
const reflected = new WispPacket({
data: this.data_,
direction:
this.direction === this.constructor.SEND ?
this.constructor.RECV :
this.direction === this.constructor.RECV ?
this.constructor.SEND :
undefined,
extra: {
reflectedFrom: this,
},
});
return reflected;
}
}
for ( const item of wisp_types ) {
WispPacket[item.label] = item;
}
const NewWispPacketStream = frameStream => {
return new ATStream({
delegate: frameStream,
transform ({ value }) {
return new WispPacket({
data: value,
direction: WispPacket.RECV,
});
},
observe ({ value }) {
// TODO: configurable behavior, or a separate stream decorator
value.log();
},
});
};
class DataBuilder {
constructor ({ leb } = {}) {
this.pos = 0;
this.steps = [];
this.leb = leb;
}
uint8 (value) {
this.steps.push(['setUint8', this.pos, value]);
this.pos++;
return this;
}
uint16 (value, leb) {
leb ??= this.leb;
this.steps.push(['setUint8', this.pos, value, leb]);
this.pos += 2;
return this;
}
uint32 (value, leb) {
leb ??= this.leb;
this.steps.push(['setUint32', this.pos, value, leb]);
this.pos += 4;
return this;
}
utf8 (value) {
const encoded = new TextEncoder().encode(value);
this.steps.push(['array', 'set', encoded, this.pos]);
this.pos += encoded.length;
return this;
}
cat (data) {
this.steps.push(['array', 'set', data, this.pos]);
this.pos += data.length;
return this;
}
build () {
const array = new Uint8Array(this.pos);
const view = new DataView(array.buffer);
for ( const step of this.steps ) {
let target = view;
let fn_name = step.shift();
if ( fn_name === 'array' ) {
fn_name = step.shift();
target = array;
}
target[fn_name](...step);
}
return array;
}
}
module.exports = {
NewCallbackByteStream,
NewVirtioFrameStream,
NewWispPacketStream,
WispPacket,
DataBuilder,
};
src/puter-wisp/test/test.js
-165
View File
@@ -1,165 +0,0 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
const assert = require('assert');
const {
NewVirtioFrameStream,
NewWispPacketStream,
WispPacket,
} = require('../src/exports');
const NewTestByteStream = uint8array => {
return (async function * () {
for ( const item of uint8array ) {
yield Uint8Array.from([item]);
}
})();
};
const NewTestFullByteStream = uint8array => {
return (async function * () {
yield uint8array;
})();
};
/**
* This will send 'sz'-sized chunks of the uint8array
* until the uint8array is exhausted. The last chunk
* may be smaller than 'sz'.
* @curry
* @param {*} sz
* @param {*} uint8array
*/
const NewTestWindowByteStream = sz => {
const fn = uint8array => {
return (async function * () {
let offset = 0;
while ( offset < uint8array.length ) {
const end = Math.min(offset + sz, uint8array.length);
const chunk = uint8array.slice(offset, end);
offset += sz;
yield chunk;
}
})();
};
fn.name_ = `NewTestWindowByteStream(${sz})`;
return fn;
};
const NewTestChunkedByteStream = chunks => {
return (async function * () {
for ( const chunk of chunks ) {
yield chunk;
}
})();
};
const test = async (name, fn) => {
console.log(`\x1B[36;1m=== [ Running test: ${name} ] ===\x1B[0m`);
await fn();
};
const BASH_TEST_BYTES = [
22, 0, 0, 0, 2, 1, 0, 0, 0, 27, 91, 63, 50, 48, 48, 52, 108, 13, 27, 91, 63, 50, 48, 48, 52, 104,
10, 0, 0, 0, 2, 1, 0, 0, 0, 40, 110, 111, 110, 101,
10, 0, 0, 0, 2, 1, 0, 0, 0, 41, 58, 47, 35, 32,
7, 0, 0, 0, 2, 1, 0, 0, 0, 13, 10,
14, 0, 0, 0, 2, 1, 0, 0, 0, 27, 91, 63, 50, 48, 48, 52, 108, 13,
17, 0, 0, 0, 2, 1, 0, 0, 0, 27, 91, 63, 50, 48, 48, 52, 104, 40, 110, 111, 110,
11, 0, 0, 0, 2, 1, 0, 0, 0, 101, 41, 58, 47, 35, 32,
];
const runit = async () => {
const stream_behaviors = [
NewTestByteStream,
NewTestFullByteStream,
NewTestWindowByteStream(2),
NewTestWindowByteStream(3),
];
for ( const stream_behavior of stream_behaviors ) {
await test(`Wisp CONTINUE ${stream_behavior.name_ ?? stream_behavior.name}`, async () => {
const byteStream = stream_behavior(Uint8Array.from([
9, 0, 0, 0, // size of frame: 9 bytes (u32-L)
3, // CONTINUE (u8)
0, 0, 0, 0, // stream id: 0 (u32-L)
0x0F, 0x0F, 0, 0, // buffer size (u32-L)
]));
const virtioStream = NewVirtioFrameStream(byteStream);
const wispStream = NewWispPacketStream(virtioStream);
const packets = [];
for await ( const packet of wispStream ) {
packets.push(packet);
}
assert.strictEqual(packets.length, 1);
const packet = packets[0];
assert.strictEqual(packet.type.id, 3);
assert.strictEqual(packet.type.label, 'CONTINUE');
assert.strictEqual(packet.type, WispPacket.CONTINUE);
});
}
await test('bash prompt chunking', async () => {
const byteStream = NewTestChunkedByteStream([
// These are data frames from virtio->twisp->bash
// "(none"
Uint8Array.from([
10, 0, 0, 0, 2, 1, 0, 0, 0,
40, 110, 111, 110, 101,
]),
// "):/# "
Uint8Array.from([
10, 0, 0, 0, 2, 1, 0, 0, 0,
41, 58, 47, 35, 32,
]),
]);
const virtioStream = NewVirtioFrameStream(byteStream);
const wispStream = NewWispPacketStream(virtioStream);
const data = [];
for await ( const packet of wispStream ) {
for ( const item of packet.payload ) {
data.push(item);
}
}
const expected = [
40, 110, 111, 110, 101,
41, 58, 47, 35, 32,
];
assert.strictEqual(data.length, expected.length);
for ( let i = 0; i < data.length; i++ ) {
assert.strictEqual(data[i], expected[i]);
}
});
};
(async () => {
try {
await runit();
} catch (e) {
console.error(e);
console.log('\x1B[31;1mTest Failed\x1B[0m');
process.exit(1);
}
console.log('\x1B[32;1mAll tests passed\x1B[0m');
})();
src/useapi/main.js
-129
View File
@@ -1,129 +0,0 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
const globalwith = (vars, fn) => {
const original_values = {};
const keys = Object.keys(vars);
for ( const key of keys ) {
if ( key in globalThis ) {
original_values[key] = globalThis[key];
}
globalThis[key] = vars[key];
}
try {
return fn();
} finally {
for ( const key of keys ) {
if ( key in original_values ) {
globalThis[key] = original_values[key];
} else {
delete globalThis[key];
}
}
}
};
const aglobalwith = async (vars, fn) => {
const original_values = {};
const keys = Object.keys(vars);
for ( const key of keys ) {
if ( key in globalThis ) {
original_values[key] = globalThis[key];
}
globalThis[key] = vars[key];
}
try {
return await fn();
} finally {
for ( const key of keys ) {
if ( key in original_values ) {
globalThis[key] = original_values[key];
} else {
delete globalThis[key];
}
}
}
};
let default_fn = () => {
const use = name => {
const parts = name.split('.');
let obj = use;
for ( const part of parts ) {
if ( ! obj[part] ) {
obj[part] = {};
}
obj = obj[part];
}
return obj;
};
const library = {
use,
def: (name, value, options = {}) => {
const parts = name.split('.');
let obj = use;
for ( const part of parts.slice(0, -1) ) {
if ( ! obj[part] ) {
obj[part] = {};
}
obj = obj[part];
}
const lastpart = parts[parts.length - 1];
if ( options.assign ) {
if ( ! obj[lastpart] ) {
obj[lastpart] = {};
}
Object.assign(obj[lastpart], value);
return;
}
obj[lastpart] = value;
},
withuse: fn => {
return globalwith({
use,
def: library.def,
}, fn);
},
awithuse: async fn => {
return await aglobalwith({
use,
def: library.def,
}, fn);
},
};
return library;
};
const useapi = function useapi () {
return default_fn();
};
// We export some things on the function itself
useapi.globalwith = globalwith;
useapi.aglobalwith = aglobalwith;
module.exports = useapi;
src/useapi/package.json
-8
View File
@@ -1,8 +0,0 @@
{
"name": "useapi",
"version": "1.0.0",
"author": "Puter Technologies Inc.",
"license": "AGPL-3.0-only",
"description": "Dynamic import interface for Puter mods",
"main": "main.js"
}
submodules/epoxy-tls
-1
Submodule submodules/epoxy-tls deleted from d6f1a8da43
submodules/twisp
-1
Submodule submodules/twisp deleted from ae6e6527d7
submodules/v86
-1
Submodule submodules/v86 deleted from d9c36dfccf
submodules/wiki
-1
Submodule submodules/wiki deleted from f6fd3b407b
tools/README.md
-43
View File
@@ -1,43 +0,0 @@
# Tools Directory
This directory contains tools for developing and running puter.
Each directory inside `/tools` is an npm workspace, so it can have its own
package.json file and dependencies.
## Scripts
### `run-selfhosted.js`
This is the main script for running a local instance of Puter.
It verifies the version of node.js you are running and attempts to explain
any errors that come up if initiating boot fails.
Puter is booted with essential modules, and modules required for local
file storage.
### `gen-release-notes.js`
Generates release notes between a hard-coded pair of versions. These versions
need to be modified manually in the script source before running.
### `check-translations.js`
Checks for missing translations in `src/gui/src/i18n/translations`
## Utilities
### `module-docgen`
Document a module.
## Libraries
### comment-parser
This is a package used by the `license-headers` tool to process existing
comments.
### file-walker
This is used by `license-headers` to walk through
source files.
tools/build_v86.sh
-45
View File
@@ -1,45 +0,0 @@
#!/bin/bash
start_dir=$(pwd)
cleanup() {
cd "$start_dir"
}
trap cleanup ERR EXIT
set -e
echo -e "\x1B[36;1m<<< Adding Targets >>>\x1B[0m"
rustup target add wasm32-unknown-unknown
rustup target add i686-unknown-linux-gnu
# Emulator assets were removed from this fork; exit early to avoid failing.
if [ ! -d "src/emulator" ]; then
echo -e "\x1B[33;1mEmulator directory missing; skipping v86 image build.\x1B[0m"
exit 0
fi
echo -e "\x1B[36;1m<<< Building v86 >>>\x1B[0m"
cd submodules/v86
make all
cd -
echo -e "\x1B[36;1m<<< Building Twisp >>>\x1B[0m"
pwd
cd submodules/epoxy-tls/server
RUSTFLAGS="-C target-feature=+crt-static" cargo +nightly b -F twisp -r --target i686-unknown-linux-gnu;
echo -e "\x1B[36;1m<<< Preparing to Build Imag >>>\x1B[0m"
cd -
cp submodules/epoxy-tls/target/i686-unknown-linux-gnu/release/epoxy-server \
src/emulator/image/assets/
echo -e "\x1B[36;1m<<< Building Image >>>\x1B[0m"
cd src/emulator/image
./clean.sh
./build.sh
cd -
tools/check-translations.js
-136
View File
@@ -1,136 +0,0 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import translations from '../src/gui/src/i18n/translations/translations.js';
import fs from 'fs';
let hadError = false;
function reportError(message) {
hadError = true;
process.stderr.write(`❌ ${message}\n`);
}
/**
* Verifies that all translation files in the translations directory are properly registered
* in the translations object. Checks for required properties like name, code, and dictionary.
* Reports errors if translations are missing, improperly configured, or have mismatched codes.
* @async
* @returns {Promise<void>}
*/
async function checkTranslationRegistrations() {
const files = await fs.promises.readdir('./src/gui/src/i18n/translations');
for (const fileName of files) {
if (!fileName.endsWith('.js')) continue;
const translationName = fileName.substring(0, fileName.length - 3);
if (translationName === 'translations') continue;
const translation = translations[translationName];
if (!translation) {
reportError(`Translation '${translationName}' is not listed in translations.js, please add it!`);
continue;
}
if (!translation.name) {
reportError(`Translation '${translationName}' is missing a name!`);
}
if (!translation.code) {
reportError(`Translation '${translationName}' is missing a code!`);
} else if (translation.code !== translationName) {
reportError(`Translation '${translationName}' has code '${translation.code}', which should be '${translationName}'!`);
}
if (typeof translation.dictionary !== 'object') {
reportError(`Translation '${translationName}' is missing a translations dictionary! Should be an object.`);
}
}
}
/**
* Validates that translation dictionaries only contain keys present in en.js
*
* Iterates through all translations (except English) and checks that each key in their
* dictionary exists in the en.js dictionary. Reports errors for any keys that don't exist.
* Skips validation if the translation dictionary is missing or invalid.
*/
function checkTranslationKeys() {
const enDictionary = translations.en.dictionary;
for (const translation of Object.values(translations)) {
// We compare against the en translation, so checking it doesn't make sense.
if (translation.code === 'en') continue;
// If the dictionary is missing, we already reported that in checkTranslationRegistrations().
if (typeof translation.dictionary !== "object") continue;
for (const [key, value] of Object.entries(translation.dictionary)) {
if (!enDictionary[key]) {
reportError(`Translation '${translation.code}' has key '${key}' that doesn't exist in 'en'!`);
}
}
}
}
/**
* Checks for usage of i18n() calls in source files and verifies that all translation keys exist in en.js
*
* Scans JavaScript files in specified source directories for i18n() function calls using regex.
* Validates that each key used in these calls exists in the English translation dictionary.
*
* @async
* @returns {Promise<void>}
*/
async function checkTranslationUsage() {
const enDictionary = translations.en.dictionary;
const sourceDirectories = [
'./src/gui/src/helpers',
'./src/gui/src/UI',
];
// Looks for i18n() calls using either ' or " for the key string.
// The key itself is at index 2 of the result.
const i18nRegex = /i18n\((['"])(.*?)\1\)/g;
for (const dir of sourceDirectories) {
const files = await fs.promises.readdir(dir, { recursive: true });
for (const relativeFileName of files) {
if (!relativeFileName.endsWith('.js')) continue;
const fileName = `${dir}/${relativeFileName}`;
const fileContents = await fs.promises.readFile(fileName, { encoding: 'utf8' });
const i18nUses = fileContents.matchAll(i18nRegex);
for (const use of i18nUses) {
const key = use[2];
if (!enDictionary.hasOwnProperty(key)) {
reportError(`Unrecognized i18n key: call ${use[0]} in ${fileName}`);
}
}
}
}
}
await checkTranslationRegistrations();
checkTranslationKeys();
await checkTranslationUsage();
if (hadError) {
process.stdout.write('Errors were found in translation files.\n');
process.exit(1);
}
process.stdout.write('✅ Translations appear valid.\n');
process.exit(0);
tools/comment-parser/main.js
-420
View File
@@ -1,420 +0,0 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
const lib = {};
lib.dedent_lines = lines => {
// If any lines are just spaces, remove the spaces
for ( let i=0 ; i < lines.length ; i++ ) {
if ( /^\s+$/.test(lines[i]) ) lines[i] = '';
}
// Remove leading and trailing blanks
while ( lines[0] === '' ) lines.shift();
while ( lines[lines.length-1] === '' ) lines.pop();
let min_indent = Number.MAX_SAFE_INTEGER;
for ( let i=0 ; i < lines.length ; i++ ) {
if ( lines[i] === '' ) continue;
let n_spaces = 0;
for ( let j=0 ; j < lines[i].length ; j++ ) {
if ( lines[i][j] === ' ' ) n_spaces++;
else break;
}
if ( n_spaces < min_indent ) min_indent = n_spaces;
}
for ( let i=0 ; i < lines.length ; i++ ) {
if ( lines[i] === '' ) continue;
lines[i] = lines[i].slice(min_indent);
}
};
/**
* Creates a StringStream object for parsing a string with position tracking
* @param {string} str - The string to parse
* @param {Object} [options] - Optional configuration object
* @param {Object} [options.state_] - Initial state with position
* @returns {Object} StringStream instance with parsing methods
*/
const StringStream = (str, { state_ } = {}) => {
const state = state_ ?? { pos: 0 };
return {
skip_whitespace () {
while ( /^\s/.test(str[state.pos]) ) state.pos++;
},
// INCOMPLETE: only handles single chars
skip_matching (items) {
while ( items.some(item => {
return str[state.pos] === item;
}) ) state.pos++;
},
fwd (amount) {
state.pos += amount ?? 1;
},
fork () {
return StringStream(str, { state_: { pos: state.pos } });
},
async get_pos () {
return state.pos;
},
async get_char () {
return str[state.pos];
},
async matches (re_or_lit) {
if ( re_or_lit instanceof RegExp ) {
const re = re_or_lit;
return re.test(str.slice(state.pos));
}
const lit = re_or_lit;
return lit === str.slice(state.pos, state.pos + lit.length);
},
async get_until (re_or_lit) {
let index;
if ( re_or_lit instanceof RegExp ) {
const re = re_or_lit;
const result = re.exec(str.slice(state.pos));
if ( ! result ) return;
index = state.pos + result.index;
} else {
const lit = re_or_lit;
const ind = str.slice(state.pos).indexOf(lit);
// TODO: parser warnings?
if ( ind === -1 ) return;
index = state.pos + ind;
}
const start_pos = state.pos;
state.pos = index;
return str.slice(start_pos, index);
},
async debug () {
const l1 = str.length;
const l2 = str.length - state.pos;
const clean = s => s.replace(/\n/, '{LF}');
return `[stream : "${
clean(str.slice(0, Math.min(6, l1)))
}"... |${state.pos}| ..."${
clean(str.slice(state.pos, state.pos + Math.min(6, l2)))
}"]`
}
};
};
const LinesCommentParser = ({
prefix
}) => {
return {
parse: async (stream) => {
stream.skip_whitespace();
const lines = [];
while ( await stream.matches(prefix) ) {
const line = await stream.get_until('\n');
if ( ! line ) return;
lines.push(line);
stream.fwd();
stream.skip_matching([' ', '\t']);
if ( await stream.get_char() === '\n' ){
stream.fwd();
break;
}
stream.skip_whitespace();
}
if ( lines.length === 0 ) return;
for ( let i=0 ; i < lines.length ; i++ ) {
lines[i] = lines[i].slice(prefix.length);
}
lib.dedent_lines(lines);
return {
lines,
};
}
};
};
const BlockCommentParser = ({
start,
end,
ignore_line_prefix,
}) => {
return {
parse: async (stream) => {
stream.skip_whitespace();
if ( ! await stream.matches(start) ) return;
stream.fwd(start.length);
const contents = await stream.get_until(end);
if ( ! contents ) return;
stream.fwd(end.length);
// console.log('ending at', await stream.debug())
const lines = contents.split('\n');
// === Formatting Time! === //
// Special case: remove the last '*' after '/**'
if ( lines[0].trim() === ignore_line_prefix ) {
lines.shift();
}
// First dedent pass
lib.dedent_lines(lines);
// If all the lines start with asterisks, remove
let allofem = true;
for ( let i=0 ; i < lines.length ; i++ ) {
if ( lines[i] === '' ) continue;
if ( ! lines[i].startsWith(ignore_line_prefix) ) {
allofem = false;
break
}
}
if ( allofem ) {
for ( let i=0 ; i < lines.length ; i++ ) {
if ( lines[i] === '' ) continue;
lines[i] = lines[i].slice(ignore_line_prefix.length);
}
// Second dedent pass
lib.dedent_lines(lines);
}
return { lines };
}
};
};
/**
* Creates a writer for line-style comments with a specified prefix
* @param {Object} options - Configuration options
* @param {string} options.prefix - The prefix to use for each comment line
* @returns {Object} A comment writer object
*/
const LinesCommentWriter = ({ prefix }) => {
return {
write: (lines) => {
lib.dedent_lines(lines);
for ( let i=0 ; i < lines.length ; i++ ) {
lines[i] = prefix + lines[i];
}
return lines.join('\n') + '\n';
}
};
};
/**
* Creates a block comment writer with specified start/end markers and prefix
* @param {Object} options - Configuration options
* @param {string} options.start - Comment start marker (e.g. "/*")
* @param {string} options.end - Comment end marker (e.g. "* /")
* @param {string} options.prefix - Line prefix within comment (e.g. " * ")
* @returns {Object} Block comment writer object
*/
const BlockCommentWriter = ({ start, end, prefix }) => {
return {
write: (lines) => {
lib.dedent_lines(lines);
for ( let i=0 ; i < lines.length ; i++ ) {
lines[i] = prefix + lines[i];
}
let s = start + '\n';
s += lines.join('\n') + '\n';
s += end + '\n';
return s;
}
};
};
/**
* Creates a new CommentParser instance for parsing and handling source code comments
*
* @returns {Object} An object with methods:
* - supports: Checks if a file type is supported
* - extract_top_comments: Extracts comments from source code
* - output_comment: Formats and outputs comments in specified style
*/
const CommentParser = () => {
const registry_ = {
object: {
parsers: {
lines: LinesCommentParser,
block: BlockCommentParser,
},
writers: {
lines: LinesCommentWriter,
block: BlockCommentWriter,
},
},
data: {
extensions: {
js: 'javascript',
cjs: 'javascript',
mjs: 'javascript',
},
languages: {
javascript: {
parsers: [
['lines', {
prefix: '//',
}],
['block', {
start: '/*',
end: '*/',
ignore_line_prefix: '*',
}],
],
writers: {
lines: ['lines', {
prefix: '// '
}],
block: ['block', {
start: '/*',
end: ' */',
prefix: ' * ',
}]
},
}
},
}
};
/**
* Gets the language configuration for a given filename by extracting and validating its extension
* @param {Object} params - The parameters object
* @param {string} params.filename - The filename to get the language for
* @returns {Object} Object containing the language configuration
*/
const get_language_by_filename = ({ filename }) => {
const { language } = (({ filename }) => {
const { language_id } = (({ filename }) => {
const { extension } = (({ filename }) => {
const components = ('' + filename).split('.');
const extension = components[components.length - 1];
return { extension };
})({ filename });
const language_id = registry_.data.extensions[extension];
if ( ! language_id ) {
throw new Error(`unrecognized language id: ` +
language_id);
}
return { language_id };
})({ filename });
const language = registry_.data.languages[language_id];
return { language };
})({ filename });
if ( ! language ) {
// TODO: use strutil quot here
throw new Error(`unrecognized language: ${language}`)
}
return { language };
}
/**
* Checks if a given filename is supported by the comment parser
* @param {Object} params - The parameters object
* @param {string} params.filename - The filename to check support for
* @returns {boolean} Whether the file type is supported
*/
const supports = ({ filename }) => {
try {
get_language_by_filename({ filename });
} catch (e) {
return false;
}
return true;
};
const extract_top_comments = async ({ filename, source }) => {
const { language } = get_language_by_filename({ filename });
// TODO: registry has `data` and `object`...
// ... maybe add `virt` (virtual), which will
// behave in the way the above code is written.
const inst_ = spec => registry_.object.parsers[spec[0]](spec[1]);
let ss = StringStream(source);
const results = [];
for (;;) {
let comment;
for ( let parser of language.parsers ) {
const parser_name = parser[0];
parser = inst_(parser);
const ss_ = ss.fork();
const start_pos = await ss_.get_pos();
comment = await parser.parse(ss_);
const end_pos = await ss_.get_pos();
if ( comment ) {
ss = ss_;
comment.type = parser_name;
comment.range = [start_pos, end_pos];
break;
}
}
// console.log('comment?', comment);
if ( ! comment ) break;
results.push(comment);
}
return results;
}
/**
* Outputs a comment in the specified style for a given filename and text
* @param {Object} params - The parameters object
* @param {string} params.filename - The filename to determine comment style
* @param {string} params.style - The comment style to use ('lines' or 'block')
* @param {string} params.text - The text content of the comment
* @returns {string} The formatted comment string
*/
const output_comment = ({ filename, style, text }) => {
const { language } = get_language_by_filename({ filename });
const inst_ = spec => registry_.object.writers[spec[0]](spec[1]);
let writer = language.writers[style];
writer = inst_(writer);
const lines = text.split('\n');
const s = writer.write(lines);
return s;
}
return {
supports,
extract_top_comments,
output_comment,
};
};
module.exports = {
StringStream,
LinesCommentParser,
BlockCommentParser,
CommentParser,
};
tools/comment-parser/package.json
-15
View File
@@ -1,15 +0,0 @@
{
"name": "comment-parser",
"version": "1.0.0",
"main": "main.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"keywords": [],
"author": "",
"license": "AGPL-3.0-only",
"description": "",
"devDependencies": {
"chai": "^5.1.1"
}
}
tools/comment-parser/test/test.js
-146
View File
@@ -1,146 +0,0 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
const {
StringStream,
LinesCommentParser,
BlockCommentParser,
CommentParser
} = require('../main');
const assert = async (label, fn) => {
if ( ! await fn() ) {
// TODO: strutil quot
throw new Error(`assert: '${label}' failed`)
}
};
describe('parsers', () => {
describe('lines-comment-parser', () => {
it ('basic test', async () => {
const parser = LinesCommentParser({ prefix: '//' });
let lines;
const ss = StringStream(`
// first line of first block
// second line of first block
// first line of second block
function () {}
`);
const results = [];
for (;;) {
comment = await parser.parse(ss);
if ( ! comment ) break;
results.push(comment.lines);
}
console.log('results?', results);
})
})
describe('block-comment-parser', () => {
it ('basic test', async () => {
const parser = BlockCommentParser({
start: '/*',
end: '*/',
ignore_line_prefix: '*',
});
let lines;
const ss = StringStream(`
/*
First block
comment
*/
/*
* second block
* comment
*/
/**
* third block
* comment
*/
function () {}
`);
const results = [];
for (;;) {
comment = await parser.parse(ss);
if ( ! comment ) break;
results.push(comment.lines);
}
console.log('results?', results);
})
it ('doesn\'t return anything for line comments', async () => {
const parser = BlockCommentParser({
start: '/*',
end: '*/',
ignore_line_prefix: '*',
});
let lines;
const ss = StringStream(`
// this comment should not be parsed
// by the block comment parser
function () {}
`);
const results = [];
for (;;) {
comment = await parser.parse(ss);
if ( ! comment ) break;
results.push(comment.lines);
}
console.log('results?', results);
})
})
describe('extract_top_comments', () => {
it ('basic test', async () => {
const parser = CommentParser();
const filename = 'test.js';
const source = `
// First lines comment
// second line of lines comment
/*
First block comment
second line of block comment
*/
`;
const results = await parser.extract_top_comments({
filename,
source,
});
console.log('results?', results);
})
})
describe('StringStream', () => {
describe('fork', () => {
it('works', async () => {
const ss = StringStream('asdf');
const ss_ = ss.fork();
ss_.fwd();
await assert('fwd worked', async () => {
return await ss_.get_char() === 's';
});
await assert('upstream state is same', async () => {
return await ss.get_char() === 'a';
});
})
})
})
});
tools/doc_helper.js
-492
View File
@@ -1,492 +0,0 @@
import fs from 'fs';
import path from 'path';
import { fileURLToPath } from 'url';
import manualOverrides from '../doc/contributors/extensions/manual_overrides.json.js';
// Get the directory name in ES modules
const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
// Create a map of manual overrides for quick lookup
const manualOverridesMap = new Map();
manualOverrides.forEach(override => {
manualOverridesMap.set(override.id, override);
});
// Array to collect all warnings
const warnings = [];
// Add a function to detect and collect duplicate events
function checkForDuplicateEvent(eventId, filePath, seenEvents) {
if (seenEvents.has(eventId)) {
const existing = seenEvents.get(eventId);
if (existing.fromManualOverride) {
warnings.push(`Event ${eventId} found in ${filePath} but already defined in manual overrides. Using manual override.`);
} else {
warnings.push(`Duplicate event ${eventId} found in ${filePath}. First seen in ${existing.filename}.`);
}
return true;
}
return false;
}
function extractEventsFromFile(filePath, seenEvents, debugMode) {
const content = fs.readFileSync(filePath, 'utf-8');
// Use a more general regex to capture all event emissions
// This captures the event name and whatever is passed as the second argument
const regex = /svc_event\.emit\(['"]([^'"]+)['"]\s*,\s*([^)]+)\)/g;
let match;
while ((match = regex.exec(content)) !== null) {
const eventName = match[1];
const eventId = eventName;
const eventArg = match[2].trim();
// Check if this file contains code that might affect event.allow
const hasAllowEffect = content.includes('event.allow') ||
content.includes('.allow =') ||
content.includes('.allow=');
// Check for duplicate events and collect warnings
if (checkForDuplicateEvent(eventId, filePath, seenEvents)) {
continue; // Skip this event if it's a duplicate
}
// Check if this event has a manual override
if (manualOverridesMap.has(eventId)) {
// Use the manual override instead of generating a new definition
const override = manualOverridesMap.get(eventId);
// Mark this as coming from manual override for later reference
override.fromManualOverride = true;
seenEvents.set(eventId, override);
continue;
}
// Generate description based on event name
let description = generateDescription(eventName);
let propertyDetails = {};
// Case 1: Inline object - extract properties directly
if (eventArg.startsWith('{')) {
// Extract properties from inline object
const propertiesMatch = eventArg.match(/{([^}]*)}/);
if (propertiesMatch) {
const propertiesText = propertiesMatch[1];
extractProperties(propertiesText, propertyDetails, hasAllowEffect, eventName);
}
}
// Case 2: Variable reference - find variable definition
else {
const varName = eventArg.trim();
// Look for variable definition patterns like: const event = { prop1: value1 };
const varDefRegex = new RegExp(`(?:const|let|var)\\s+${varName}\\s*=\\s*{([^}]*)}`, 'g');
let varMatch;
if ((varMatch = varDefRegex.exec(content)) !== null) {
const propertiesText = varMatch[1];
extractProperties(propertiesText, propertyDetails, hasAllowEffect, eventName);
}
}
// Add the event to our collection
seenEvents.set(eventId, {
id: eventId,
event: eventName,
filename: path.basename(filePath),
description: description,
properties: propertyDetails,
fromManualOverride: false
});
}
}
// Helper function to extract properties from a properties text string
function extractProperties(propertiesText, propertyDetails, hasAllowEffect, eventName) {
// filter out all comments (lines starting with //)
const lines = propertiesText.split('\n').map(line => line.trim()).filter(line => !line.startsWith('//'));
// glue all lines together, then split by commas
const gluedTest = lines.join('\n');
const properties = gluedTest
.split(/\s*,\s*/)
.map(prop => prop.split(/[^_A-Za-z0-9]/)[0].trim())
.filter(prop => prop);
// // const event = { allow: true, email };
// // text: allow: true, email
// // split to: [allow: true] [email]
// const properties = propertiesText
// .split(/\s*,\s*/)
// .map(prop => prop.split(':')[0].trim())
// .filter(prop => prop);
// Generate property details
properties.forEach(prop => {
propertyDetails[prop] = {
type: guessType(prop),
mutability: hasAllowEffect ? 'effect' : 'no-effect',
summary: guessSummary(prop, eventName)
};
});
}
function generateDescription(eventName) {
const parts = eventName.split('.');
if (parts.length >= 2) {
const system = parts[0];
const action = parts.slice(1).join('.');
if (action.includes('create')) {
return `This event is emitted when a ${parts[parts.length - 1]} is created.`;
} else if (action.includes('update') || action.includes('write')) {
return `This event is emitted when a ${parts[parts.length - 1]} is updated.`;
} else if (action.includes('delete') || action.includes('remove')) {
return `This event is emitted when a ${parts[parts.length - 1]} is deleted.`;
} else if (action.includes('progress')) {
return `This event reports progress of a ${parts[parts.length - 1]} operation.`;
} else if (action.includes('validate')) {
return `This event is emitted when a ${parts[parts.length - 1]} is being validated.\nThe event can be used to block certain ${parts[parts.length - 1]}s from being validated.`;
} else {
return `This event is emitted for ${system} ${action.replace(/[-\.]/g, ' ')} operations.`;
}
}
return `This event is emitted for ${eventName} operations.`;
}
function guessType(propertyName) {
// Guess the type based on property name
if (propertyName === 'node') return 'FSNodeContext';
if (propertyName === 'context') return 'Context';
if (propertyName === 'user') return 'User';
if (propertyName.includes('path')) return 'string';
if (propertyName.includes('id')) return 'string';
if (propertyName.includes('name')) return 'string';
if (propertyName.includes('progress')) return 'number';
if (propertyName.includes('tracker')) return 'ProgressTracker';
if (propertyName.includes('meta')) return 'object';
if (propertyName.includes('policy')) return 'Policy';
if (propertyName.includes('allow')) return 'boolean';
return 'any';
}
function guessSummary(propertyName, eventName) {
// Generate summary based on property name and event context
if (propertyName === 'node') {
const entityType = eventName.split('.').pop();
return `the ${entityType} that was affected`;
}
if (propertyName === 'context') return 'current context';
if (propertyName === 'user') return 'user associated with the operation';
if (propertyName.includes('path')) return 'path to the affected resource';
if (propertyName.includes('tracker')) return 'tracks progress of the operation';
if (propertyName.includes('meta')) return 'additional metadata for the operation';
if (propertyName.includes('policy')) return 'policy information for the operation';
if (propertyName.includes('allow')) return 'whether the operation is allowed';
// Default summary based on property name
return propertyName.replace(/_/g, ' ');
}
function scanDirectory(directory, seenEvents, debugMode) {
const files = fs.readdirSync(directory);
for (const file of files) {
const filePath = path.join(directory, file);
const stat = fs.statSync(filePath);
if (stat.isDirectory()) {
scanDirectory(filePath, seenEvents, debugMode);
} else if (file.endsWith('.js')) {
try {
extractEventsFromFile(filePath, seenEvents, debugMode);
} catch (error) {
warnings.push(`Error processing file ${filePath}: ${error.message}`);
}
}
}
}
function generateTestExtension(events) {
let code = `// Test extension for event listeners\n\n`;
events.forEach(event => {
const eventId = event.id;
const eventName = event.event ? event.event.toUpperCase() : eventId.split('.').slice(1).join('.').toUpperCase();
code += `extension.on('${eventId}', event => {\n`;
code += ` console.log('GOT ${eventName} EVENT', event);\n`;
code += `});\n\n`;
});
return code;
}
function main() {
const args = process.argv.slice(2);
if (args.length < 1) {
console.error('Usage: node doc_helper.js <directory> [output_file] [--generate-test] [--test-dir=<directory>] [--debug]');
// node tools/doc_helper.js . doc/contributors/extensions/events.json.js
// [output_file] [--generate-test] [--test-dir=<directory>] [--debug]');
process.exit(1);
}
// Resolve directory path relative to project root
const directory = path.resolve(path.join(path.dirname(__dirname), args[0]));
let outputFile = null;
let generateTest = false;
let testOutputDir = "./extensions/";
let debugMode = false;
// Parse arguments
for (let i = 1; i < args.length; i++) {
if (args[i] === '--generate-test') {
generateTest = true;
} else if (args[i].startsWith('--test-dir=')) {
testOutputDir = args[i].substring('--test-dir='.length);
} else if (args[i] === '--debug') {
debugMode = true;
} else if (!args[i].startsWith('--')) {
// Only treat non-flag arguments as output file
outputFile = path.resolve(path.join(path.dirname(__dirname), args[i]));
}
}
// Resolve test output directory relative to project root if it's not an absolute path
if (!path.isAbsolute(testOutputDir)) {
testOutputDir = path.resolve(path.join(path.dirname(__dirname), testOutputDir));
}
const seenEvents = new Map();
// First, add all manual overrides to the seenEvents map
manualOverrides.forEach(override => {
// Mark this as coming from manual override for later reference
override.fromManualOverride = true;
seenEvents.set(override.id, override);
});
// Then scan the directory for additional events
scanDirectory(directory, seenEvents, debugMode);
// Check for any manual overrides that weren't used
manualOverrides.forEach(override => {
const event = seenEvents.get(override.id);
if (!event || !event.fromManualOverride) {
warnings.push(`Manual override for ${override.id} exists but no matching event was found in the codebase.`);
}
});
const result = Array.from(seenEvents.values());
// Sort events alphabetically by ID
result.sort((a, b) => a.id.localeCompare(b.id));
// Format the output to match events.json.js
const formattedOutput = formatEventsOutput(result);
// Output the result
if (outputFile) {
fs.writeFileSync(outputFile, formattedOutput);
console.log(`Event metadata written to ${outputFile}`);
} else {
console.log(formattedOutput);
}
// Generate test extension file if requested
if (generateTest) {
const testCode = generateTestExtension(result);
// Ensure the output directory exists
if (!fs.existsSync(testOutputDir)) {
fs.mkdirSync(testOutputDir, { recursive: true });
}
const testFilePath = path.join(testOutputDir, 'testex.js');
fs.writeFileSync(testFilePath, testCode);
console.log(`Test extension file generated: ${testFilePath}`);
}
// Print warnings in the requested format
if (warnings.length > 0) {
// Collect duplicate events
const duplicateEvents = new Set();
const overrideEvents = new Set();
const otherWarnings = [];
warnings.forEach(warning => {
if (warning.includes("Duplicate event")) {
// Extract event ID from the warning message
const match = warning.match(/Duplicate event (core\.[^ ]+)/);
if (match && match[1]) {
duplicateEvents.add(match[1]);
}
} else if (warning.includes("already defined in manual overrides")) {
// Extract event ID from the warning message
const match = warning.match(/Event (core\.[^ ]+) found/);
if (match && match[1]) {
overrideEvents.add(match[1]);
}
} else {
otherWarnings.push(warning);
}
});
// Output in the requested format
console.log(`\nduplicate events: ${Array.from(duplicateEvents).join(', ')}`);
console.log(`Override events: ${Array.from(overrideEvents).join(', ')}`);
// If there are any other warnings, print them too
if (otherWarnings.length > 0) {
console.log("\nOther warnings:");
otherWarnings.forEach(warning => {
console.log(`- ${warning}`);
});
}
}
}
/**
* Format the events data to match the events.json.js format
*/
function formatEventsOutput(events) {
let output = 'export default [\n';
events.forEach((event, index) => {
// Check if this is a manual override
if (event.fromManualOverride) {
// This is a manual override, output it exactly as defined
output += ' {\n';
output += ` id: '${event.id}',\n`;
output += ` description: \``;
// Format the description with proper indentation, preserving original formatting
// Don't add extra newlines before or after the description
output += event.description;
output += `\`,\n`;
// Add properties if they exist, preserving exact format
if (event.properties && Object.keys(event.properties).length > 0) {
output += ' properties: {\n';
Object.entries(event.properties).forEach(([propName, propDetails], propIndex) => {
output += ` ${propName}: {\n`;
output += ` type: '${propDetails.type}',\n`;
output += ` mutability: '${propDetails.mutability}',\n`;
output += ` summary: '${propDetails.summary}'`;
// Add notes array if it exists
if (propDetails.notes && propDetails.notes.length > 0) {
output += `,\n notes: [\n`;
propDetails.notes.forEach((note, noteIndex) => {
output += ` '${note}'`;
if (noteIndex < propDetails.notes.length - 1) {
output += ',';
}
output += '\n';
});
output += ` ]`;
}
output += '\n }';
// Add comma if not the last property
if (propIndex < Object.keys(event.properties).length - 1) {
output += ',';
}
output += '\n';
});
output += ' },\n';
}
// Add example if it exists
if (event.example) {
output += ' example: {\n';
output += ` language: '${event.example.language}',\n`;
output += ` code: /*${event.example.language}*/\``;
// Preserve the exact formatting of the example code
// Don't add extra newlines and preserve escape sequences exactly as they are
output += event.example.code;
output += `\`\n`;
output += ' },\n';
}
output += ' }';
} else {
// This is an auto-generated event
output += ' {\n';
output += ` id: '${event.id}',\n`;
output += ` description: \`\n`;
// Format the description with proper indentation
const descriptionLines = event.description.split('\n');
descriptionLines.forEach(line => {
output += ` ${line}\n`;
});
output += ` \`,\n`;
// Add properties if they exist
if (Object.keys(event.properties).length > 0) {
output += ' properties: {\n';
Object.entries(event.properties).forEach(([propName, propDetails], propIndex) => {
output += ` ${propName}: {\n`;
output += ` type: '${propDetails.type}',\n`;
output += ` mutability: '${propDetails.mutability === 'effect' ? 'mutable' : 'no-effect'}',\n`;
output += ` summary: '${propDetails.summary}',\n`;
// Add notes array with appropriate content
if (propName === 'allow' && event.event.includes('validate')) {
output += ` notes: [\n`;
output += ` 'If set to false, the ${event.event.split('.')[0]} will be considered invalid.',\n`;
output += ` ],\n`;
} else if (propName === 'email' && event.event.includes('validate')) {
output += ` notes: [\n`;
output += ` 'The email may have already been cleaned.',\n`;
output += ` ],\n`;
} else {
output += ` notes: [],\n`;
}
output += ' }';
// Add comma if not the last property
if (propIndex < Object.keys(event.properties).length - 1) {
output += ',';
}
output += '\n';
});
output += ' },\n';
}
output += ' }';
}
// Add comma if not the last event
if (index < events.length - 1) {
output += ',';
}
output += '\n';
});
output += '];\n';
return output;
}
main();
// Updated Sun Mar 9 23:52:51 EDT 2025
tools/file-walker/package.json
-15
View File
@@ -1,15 +0,0 @@
{
"name": "file-walker",
"version": "1.0.0",
"description": "",
"main": "test.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"keywords": [],
"author": "",
"license": "AGPL-3.0-only",
"dependencies": {
}
}
tools/file-walker/test.js
-208
View File
@@ -1,208 +0,0 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
const fs = require('fs');
const fsp = fs.promises;
const path_ = require('path');
const EXCLUDE_LISTS = {
NOT_SOURCE: [
/^\.git/,
/^volatile\//,
/^node_modules\//,
/\/node_modules$/,
/^node_modules$/,
/package-lock\.json/,
/^src\/dev-center\/js/,
/src\/backend\/src\/public\/assets/,
/^src\/gui\/src\/lib/,
/^eslint\.config\.js$/,
]
};
EXCLUDE_LISTS.NOT_AGPL = [
...EXCLUDE_LISTS.NOT_SOURCE,
/^src\/puter-js/,
];
const hl_readdir = async path => {
const names = await fs.promises.readdir(path);
const entries = [];
for ( const name of names ) {
// wet: copied from phoenix shell
const stat_path = path_.join(path, name);
const stat = await fs.promises.lstat(stat_path);
entries.push({
name,
is_dir: stat.isDirectory(),
is_symlink: stat.isSymbolicLink(),
symlink_path: stat.isSymbolicLink() ? await fs.promises.readlink(stat_path) : null,
size: stat.size,
modified: stat.mtimeMs / 1000,
created: stat.ctimeMs / 1000,
accessed: stat.atimeMs / 1000,
mode: stat.mode,
uid: stat.uid,
gid: stat.gid,
});
}
return entries;
};
const walk = async function* walk (options, root_path, components = []) {
const current_path = path_.join(root_path, ...components);
const entries = await hl_readdir(current_path);
outer:
for ( const entry of entries ) {
entry.dirpath = current_path;
entry.path = path_.join(current_path, entry.name);
// TODO: labelled break?
for ( const exclude_regex of (options.excludes ?? []) ) {
if ( exclude_regex.test(entry.path) ) {
continue outer;
}
}
if ( ! options.pre_order ) yield entry;
if ( entry.is_dir ) {
yield* walk(options, root_path, [...components, entry.name]);
}
if ( options.pre_order ) yield entry;
}
};
const modes = {
primary_source_files: {
excludes: [
]
},
};
const util = require('util');
const exec = util.promisify(require('child_process').exec);
async function git_blame(path) {
const abs_path = path_.resolve(path);
try {
const { stdout } = await exec(`git blame -f "${abs_path}"`, {
maxBuffer: 1024 * 1024
});
const blameLines = stdout.split('\n');
const parsedBlame = blameLines
.map(line => {
if (!line.trim()) return null;
// console.log(line);
const parts = line.split(/\s+/);
let [commitHash, path, author, timestamp, lineNumber, , ,] = parts;
author = author.slice(1);
const o = {
commitHash,
author,
timestamp,
lineNumber: parseInt(lineNumber, 10),
};
return o;
})
.filter(item => item !== null)
;
return parsedBlame;
} catch (error) {
console.log('AZXV')
throw new Error(`Error executing git blame: ${error.message}`);
}
}
// Example usage
const blame = async (path) => {
try {
const result = await git_blame(path);
// console.log('result?', result)
return result;
} catch ( e ) {
console.log('SKIPPED: ' + e.message);
}
return [];
}
const walk_test = async () => {
// console.log(await hl_readdir('.'));
for await ( const value of walk({
excludes: EXCLUDE_LISTS.NOT_SOURCE,
}, '.') ) {
if ( ! value.is_dir ) continue;
console.log('value', value.path);
}
}
const authors = {};
const blame_test = async () => {
// const results = await blame('src/backend/src/services/HostDiskUsageService.js');
// const results = await blame('package.json');
console.log('results', results)
return;
for ( const result of results ) {
if ( ! authors[result.author] ) {
authors[result.author] = { lines: 0 };
}
authors[result.author].lines++;
}
console.log('AUTHORS', authors);
}
/*
Contribution count function to test file walking and
git blame parsing.
*/
const walk_and_blame = async () => {
// console.log(await hl_readdir('.'));
for await ( const value of walk({
excludes: EXCLUDE_LISTS.NOT_SOURCE,
}, '.') ) {
if ( value.is_dir ) continue;
console.log('value', value.path);
const results = await blame(value.path);
for ( const result of results ) {
if ( ! authors[result.author] ) {
authors[result.author] = { lines: 0 };
}
authors[result.author].lines++;
}
}
console.log('AUTHORS', authors);
}
if ( require.main === module ) {
const main = walk_and_blame;
main();
}
module.exports = {
walk,
EXCLUDE_LISTS,
};
tools/gen-release-notes.js
-249
View File
@@ -1,249 +0,0 @@
/*
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { simpleGit } from 'simple-git';
// GitHub repository URL for generating commit links in release notes
const REPO_URL = 'https://github.com/HeyPuter/puter';
const params = {
from: 'v2.5.0',
to: 'v2.5.1',
date: '2025-02-13',
};
const git = simpleGit();
const log = await git.log({ from: params.from });
const commits = log.all;
// Array of all commits from git log between specified versions
const CC_REGEX = /^([a-z0-9]+)(\([a-z0-9]+\))?:\s(.*)/;
const parse_conventional_commit = message => {
const parts = CC_REGEX.exec(message);
if ( ! parts ) return null;
let [match, type, scope, summary] = parts;
if ( ! match ) return null;
if ( scope ) scope = scope.slice(1, -1);
return { type, scope, summary };
};
const types = {
feat: {
label: 'Features'
},
i18n: {
label: 'Translations'
},
fix: {
label: 'Bug Fixes'
},
};
const scopes = {
puter: {
label: 'Puter'
},
phoenix: {
label: 'Phoenix Shell'
},
git: {
label: 'Puter Git'
},
backend: {
label: 'Backend'
},
api: {
label: 'API',
},
gui: {
label: 'GUI'
},
puterjs: {
label: 'Puter JS'
},
tools: {
ignore: true,
},
security: {
label: 'Security',
},
ai: {
label: 'AI',
},
putility: {
label: 'Putility',
},
docker: {
label: 'Docker',
},
};
const scope_aliases = {
main: 'puter',
ui: 'gui',
parsely: 'phoenix',
};
const complicated_cases = [
/**
* Handles special cases for commit message transformations
* @type {Array<function>}
*/
function fix_i18n ({ commit, meta }) {
if ( meta.scope === 'i18n' ) {
meta.type = 'i18n';
meta.scope = undefined;
}
},
function deps_scope ({ commit, meta }) {
if ( meta.scope === 'deps' ) {
meta.type = 'chore';
meta.scope = undefined;
}
},
function puterai_is_ai ({ commit, meta }) {
const ai_scopes = ['puterai', 'puerai', 'puter-ai'];
if ( ai_scopes.includes(meta.scope) ) {
meta.scope = 'ai';
}
},
function doc_scopes ({ commit, meta }) {
const doc_scopes = ['readme'];
if ( doc_scopes.includes(meta.scope) ) {
meta.type = 'doc';
meta.scope = undefined;
}
}
];
const retro_prefixes_0 = {
i18n: [
'883601142873f10d69c84874499065a7d29af054',
'17145d0be6a9a1445947cc0c4bec8f16a475144c',
'e61039faf409b0ad85c7513b0123f3f2e92ebe32',
'bffa192805216fc17045cd8d629f34784dca7f3f',
'fe5be7f3cf7f336730137293ba86a637e8d8591d',
'78a0acea6980b6d491da4874edbd98e17c0d9577',
'a96abb5793528d0dc56d75f95d771e1dcf5960d1',
'f5a8ee1c6ab950d62c90b6257791f026a508b4e4',
'47ec74f0aa6adb3952e6460909029a4acb0c3039',
'473b6512c697854e3f3badae1eb7b87742954da5',
'8440f566b91c9eb4f01addcb850061e3fbe3afc7',
'92abc9947f811f94f17a5ee5a4b73ee2b210900a',
'cff488f4f4378ca6c7568a585a665f2a3b87b89c',
'3b8af7cc5c1be8ed67be827360bbfe0f0b5027e9',
'84e31eff2f58584d8fab7dd10606f2f6ced933a2',
'81781f80afc07cd1e6278906cdc68c8092fbfedf',
'56820cf6ee56ff810a6b495a281ccbb2e7f9d8fb',
'69a80ab3d2c94ee43d96021c3bcbdab04a4b5dc6',
'8e297cd7e30757073e2f96593c363a273b639466',
'151527825f1eb4b060aaf97feb7d18af4fcddbf2',
'8bece96f6224a060d5b408e08c58865fadb8b79c',
'333d6e3b651e460caca04a896cbc8c175555b79b',
'8a3d0430f39f872b8a460c344cce652c340b700b',
'b9e73b7288aebb14e6bbf1915743e9157fc950b1',
'c2d3d69dbe33f36fcae13bcbc8e2a31a86025af9',
'382fb24dbb1737a8a54ed2491f80b2e2276cde61',
],
fix: [
'535475b3c36a37e3319ed067a24fb671790dcda3',
'45f131f8eaf94cf3951ca7ffeb6f311590233b8a',
'02e1b1e8f5f8e22d7ab39ebff99f7dd8e08a4221',
],
doc: [
'338004474f078a00608af1d0ebf8a7f9534bad28',
'6c4c73a9e85ff8eb5e7663dcce11f4d1f824032b',
'c19c18bfcf163b37e3d173b8fa50393dfb9f540f',
],
feat: [
'8e7306c23be01ee6c31cdb4c99f2fb1f71a2247f',
],
meta: [
'b3c1b128e2d8519bc816cdcd3220c8f40e05bb01',
'452b7495b1736df90bc748dbf818407488875754',
],
};
const message_changes = {
'1f7f094282fae915a2436701cfb756444cd3f781': 'feat: add new file templates',
'64e4299ac0a4c9e1de7a9d089e2d7529a9530818': 'doc: docker instructions for Windows',
'f897e844989083b0b369ba0ce4d2c5a9f3db5ad8': 'fix: #432',
};
const retro_prefixes = {};
for ( const prefix in retro_prefixes_0 ) {
for ( const commit_hash of retro_prefixes_0[prefix] ) {
console.log('PREFIX', commit_hash, prefix);
retro_prefixes[commit_hash] = prefix;
}
}
const data = {};
const ensure_scope = name => {
if ( data[name] ) return;
const o = data[name] = {};
for ( const k in types ) o[k] = [];
};
for ( const commit of commits ) {
if ( message_changes.hasOwnProperty(commit.hash) ) {
commit.message = message_changes[commit.hash];
}
if ( retro_prefixes.hasOwnProperty(commit.hash) ) {
commit.message = retro_prefixes[commit.hash] + ': ' +
commit.message;
}
const meta = parse_conventional_commit(commit.message);
if ( ! meta ) continue;
for ( const transformer of complicated_cases ) {
transformer({ commit, meta });
}
let scope = meta.scope ?? 'puter';
while ( scope in scope_aliases ) {
scope = scope_aliases[scope];
}
if ( ! scopes[scope] ) {
console.log(commit);
throw new Error(`missing scope: ${scope}`);
}
if ( scopes[scope].ignore ) continue;
ensure_scope(scope);
if ( types.hasOwnProperty(meta.type) ) {
data[scope][meta.type].push({ meta, commit });
}
}
let s = '';
s += `## ${params.to} (${params.date})\n\n`;
for ( const scope_name in data ) {
const scope = data[scope_name];
s += `### ${scopes[scope_name].label}\n\n`;
for ( const type_name in types ) {
const type = types[type_name];
const items = scope[type_name];
if ( items.length == 0 ) continue;
s += `\n#### ${type.label}\n\n`;
for ( const { meta, commit } of items ) {
const shorthash = commit.hash.slice(0,7)
s += `- ${meta.summary} ([${shorthash}](${REPO_URL}/commit/${commit.hash}))\n`;
}
}
}
console.log(s);
tools/genwiki/main.js
-231
View File
@@ -1,231 +0,0 @@
const { walk, EXCLUDE_LISTS } = require('../file-walker/test');
const fs = require('fs').promises;
const path_ = require('node:path');
const FILE_EXCLUDES = [
/(^|\/)\.git/,
/^volatile\//,
/^node_modules\//,
/\/node_modules$/,
/^submodules\//,
/^node_modules$/,
/package-lock\.json/,
/^src\/dev-center\/js/,
/src\/backend\/src\/public\/assets/,
/^src\/gui\/src\/lib/,
/^eslint\.config\.js$/,
// translation readme copies
/(^|\/)doc\/i18n/,
// irrelevant documentation
/(^|\/)doc\/graveyard/,
// development logs
/\/devlog\.md$/,
]
const ROOT_DIR = path_.join(__dirname, '../..');
const WIKI_DIR = path_.join(__dirname, '../../submodules/wiki');
const path_to_name = path => {
// Special case for Home.md
if ( path === 'doc/README.md' ) return 'Home';
// Remove src/ and doc/ components
// path = path.replace(/src\//g, '')
path = path.replace(/doc\//g, '')
// Hyphenate components
path = path.replace(/-/g, '_')
path = path.replace(/\//g, '-')
// Remove extension
path = path.replace(/\.md$/, '')
return path;
}
const fix_relative_links = (content, entry) => {
const originalDir = path_.dirname(entry);
// Markdown links: [text](path/to/file.md), [text](path/to/file#section), etc
return content.replace(/\[([^\]]+)\]\(([^)]+)\)/g, (match, text, link) => {
// Skip external links
if (link.startsWith('http://') || link.startsWith('https://') || link.startsWith('/')) {
return match;
}
// Anchor links within the same file aren't changed
if (link.startsWith('#')) return match;
// Split the link to separate the path from the anchor
const [linkPath, anchor] = link.split('#');
// Resolve the relative path
let resolvedPath = path_.normalize(path_.join(originalDir, linkPath));
// Find the matching wiki path
const wikiPath = path_to_name(resolvedPath);
const newLink = anchor ? `${wikiPath}#${anchor}` : wikiPath;
return `[${text}](${newLink})`;
});
};
const main = async () => {
const walk_iter = walk({
excludes: FILE_EXCLUDES,
}, ROOT_DIR);
const documents = [];
for await ( const value of walk_iter ) {
let path = value.path;
path = path_.relative(ROOT_DIR, path);
// File must be under a doc/ directory
if ( ! path.match(/(^|\/)doc\//) ) continue;
// File must be markdown
if ( ! path.match(/\.md/) ) continue;
let outputName = path_to_name(path);
// Read file content
let content = await fs.readFile(value.path, 'utf8');
// Get the first heading from the file to use as title
const titleMatch = content.match(/^#\s+(.+)$/m);
const title = titleMatch ? titleMatch[1] : outputName.replace(/-/g, ' ');
// Fix internal links
content = fix_relative_links(content, path);
// Write the modified content to the wiki directory
await fs.writeFile(path_.join(WIKI_DIR, outputName + '.md'), content);
// Store information for sidebar
const sidebarPath = outputName.split('-');
// The original path structure (minus doc/) helps determine the hierarchy
documents.push({
sidebarPath,
outputName,
title: title
});
}
// Generate _Sidebar.md
const sidebarContent = generate_sidebar(documents);
await fs.writeFile(path_.join(WIKI_DIR, '_Sidebar.md'), sidebarContent);
}
const format_name = name => {
if ( name === 'api' ) return 'API';
if ( name === 'contributors' ) return 'For Contributors';
return name.charAt(0).toUpperCase() + name.slice(1);
}
const generate_sidebar = (documents) => {
// Sort entries by path to group related files together
documents.sort((a, b) => {
const pathA = a.sidebarPath.slice(0, -1).join('/');
const pathB = b.sidebarPath.slice(0, -1).join('/');
if ( pathA !== pathB ) {
return pathA.localeCompare(pathB);
}
// README.md always goes first
const isReadmeA = a.outputName.toLowerCase().includes('readme') ||
a.outputName.toLowerCase().includes('home');
const isReadmeB = b.outputName.toLowerCase().includes('readme') ||
b.outputName.toLowerCase().includes('home');
if (isReadmeA) return -1;
if (isReadmeB) return 1;
return a.title.localeCompare(b.title);
});
// Format a document link the same way everywhere
const formatDocumentLink = (document) => {
let title = document.title;
if ( document.outputName.split('-').slice(-1)[0].toLowerCase() === 'readme' ) {
title = 'Index (README.md)';
}
if ( document.outputName.split('-').slice(-1)[0].toLowerCase() === 'home' ) {
title = `Home`;
}
return `* [${title}](${document.outputName.replace('.md', '')})\n`;
};
// Recursive function to build sidebar sections
const buildSection = (docs, depth = 0, prefix = '') => {
let result = '';
const directDocs = [];
const subSections = new Map();
// Separate direct documents from those in subsections
for (const doc of docs) {
if (doc.sidebarPath.length <= depth + 1) {
// Direct document at this level
directDocs.push(doc);
} else {
// Document belongs in a subsection
const sectionName = doc.sidebarPath[depth];
if (!subSections.has(sectionName)) {
subSections.set(sectionName, []);
}
subSections.get(sectionName).push(doc);
}
}
// Add direct documents
for (const doc of directDocs) {
result += formatDocumentLink(doc);
}
// Process subsections recursively
for (const [sectionName, sectionDocs] of subSections.entries()) {
// Generate heading with appropriate level
const headingLevel = '#'.repeat(depth + 2);
const formattedName = format_name(sectionName)
result += `\n${headingLevel} ${formattedName}\n`;
// Process the subsection documents
result += buildSection(sectionDocs, depth + 1, `${prefix}${sectionName}/`);
}
return result;
};
// Start with the main heading
let sidebar = "## General\n\n";
// Split documents into top-level and those in sections
const topLevelDocs = documents.filter(doc => doc.sidebarPath.length <= 1);
const sectionDocs = documents.filter(doc => doc.sidebarPath.length > 1);
// Add top-level documents
for (const doc of topLevelDocs) {
sidebar += formatDocumentLink(doc);
}
// Group the remaining documents by their top-level sections
const topLevelSections = new Map();
for (const doc of sectionDocs) {
const sectionName = doc.sidebarPath[0];
if (!topLevelSections.has(sectionName)) {
topLevelSections.set(sectionName, []);
}
topLevelSections.get(sectionName).push(doc);
}
// Process each top-level section
for (const [sectionName, sectionDocs] of topLevelSections.entries()) {
const formattedName = format_name(sectionName);
sidebar += `\n## ${formattedName}\n`;
sidebar += buildSection(sectionDocs, 1, `${sectionName}/`);
}
return sidebar;
};
main();
tools/genwiki/package.json
-11
View File
@@ -1,11 +0,0 @@
{
"name": "genwiki",
"version": "0.0.0",
"description": "Generate github wiki",
"main": "main.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "Puter Technologies Inc.",
"license": "AGPL-3.0-only"
}