Compare commits

..

6 Commits

Author SHA1 Message Date
calebboyd 2939f07eeb chore: bump resolve-dependencies 2020-06-30 17:31:28 -05:00
calebboyd 00f65e9b91 chore: bump resolve-dependencies node 10 compat 2020-06-26 12:07:39 -05:00
calebboyd 58f276fae6 chore: bump resolve-dependencies 2020-06-26 11:42:23 -05:00
calebboyd 7c539db738 chore: bump resolve-dependencies 2020-06-26 10:26:53 -05:00
calebboyd ae51cc6b87 fix: set explicit release 2020-06-21 20:16:45 -05:00
calebboyd cf23147c9c fix: windows build 10.16 2019-06-24 15:57:54 -05:00
41 changed files with 2758 additions and 8680 deletions
-1
View File
@@ -1 +0,0 @@
custom: https://www.paypal.me/calebsboyd
-23
View File
@@ -1,23 +0,0 @@
---
name: Bug Report
about: Report a bug encountered while running Nexe
labels: bug
---
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks!
-->
**What happened**:
**What you expected to happen**:
**How to reproduce it (as minimally and precisely as possible)**:
**Anything else we need to know?**:
**Environment**:
- Platform(OS/Version):
- Host Node Version:
- Target Node Version:
- Nexe version:
- Python Version:
-11
View File
@@ -1,11 +0,0 @@
---
name: Feature Request
about: Suggest an feature request
labels: feature
---
<!-- Please only use this template for submitting feature requests -->
**What would you like to be added**:
**Why is this needed**:
-11
View File
@@ -1,11 +0,0 @@
**What this PR does / why we need it**:
**Which issue(s) this PR fixes**:
<!--
*Automatically closes linked issue when PR is merged.
Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`.
_If PR is about `failing-tests or flakes`, please post the related issues/tests in a comment and do not use `Fixes`_*
-->
Fixes #
**Special notes for your reviewer**:
@@ -1,21 +0,0 @@
name: No Response
# Both `issue_comment` and `scheduled` event types are required for this Action
# to work properly.
on:
issue_comment:
types: [created]
schedule:
# 9AM Central
- cron: '0 14 * * *'
jobs:
noResponse:
runs-on: ubuntu-latest
steps:
- uses: lee-dohm/no-response@v0.5.0
with:
token: ${{ github.token }}
closeComment: 'Closed awaiting response from OP, please open a new issue with the requested information and/or examples'
daysUntilClose: 21
responseRequiredLabel: more-information-needed
+21 -55
View File
@@ -1,4 +1,3 @@
<p align="center"><img src="https://cloud.githubusercontent.com/assets/2391349/23598327/a17bb68a-01ee-11e7-8f55-88a5fc96e997.png" /></p> <p align="center"><img src="https://cloud.githubusercontent.com/assets/2391349/23598327/a17bb68a-01ee-11e7-8f55-88a5fc96e997.png" /></p>
<p align="center"> <p align="center">
@@ -6,10 +5,9 @@
<a href="https://www.npmjs.com/package/nexe"><img src="https://img.shields.io/npm/dt/nexe.svg" alt="Downloads"></a> <a href="https://www.npmjs.com/package/nexe"><img src="https://img.shields.io/npm/dt/nexe.svg" alt="Downloads"></a>
<a href="https://www.npmjs.com/package/nexe"><img src="https://img.shields.io/npm/v/nexe.svg" alt="Version"></a> <a href="https://www.npmjs.com/package/nexe"><img src="https://img.shields.io/npm/v/nexe.svg" alt="Version"></a>
<a href="https://www.npmjs.com/package/nexe"><img src="https://img.shields.io/npm/l/nexe.svg" alt="License"></a> <a href="https://www.npmjs.com/package/nexe"><img src="https://img.shields.io/npm/l/nexe.svg" alt="License"></a>
<a href="https://discord.gg/2qTH52zNcZ"><img src="https://discordapp.com/api/guilds/814334925049561168/widget.png?style=shield"></a>
</p> </p>
<p align="center">Install:&nbsp<code>npm i nexe -g</code></p> <p align="center"><code>npm i nexe -g</code></p>
<p align="center">Nexe is a command-line utility that compiles your Node.js application into a single executable file.</p> <p align="center">Nexe is a command-line utility that compiles your Node.js application into a single executable file.</p>
<p align="center"> <p align="center">
@@ -49,39 +47,10 @@ For more CLI options see: `nexe --help`
Additional files or resources can be added to the binary by passing `-r "glob/pattern/**/*"`. These included files can be read in the application by using `fs.readFile` or `fs.readFileSync`. Additional files or resources can be added to the binary by passing `-r "glob/pattern/**/*"`. These included files can be read in the application by using `fs.readFile` or `fs.readFileSync`.
## Compiling the nexe Executable ## Compiling Node
By default `nexe` will attempt to download a pre-built executable. These are listed on the [releases page](https://github.com/nexe/nexe/releases/tag/v3.3.3). The exact version you want may be unavailable or you may want to customize what is built. See `nexe --help` for a list of options available when passing the [`--build`](#build-boolean) option. You will also need to ensure your environment is setup to [build node](https://github.com/nodejs/node/blob/master/BUILDING.md). Note: the `python` binary in your path should be an acceptable version of python 3; you can create a [symlink](https://github.com/nexe/nexe/issues/354#issuecomment-319874486) or use the `--python` parameter (e.g. `nexe --build --python=$(which python3)`). By default `nexe` will attempt to download a pre-built executable. However, It may be unavailable ([github releases](https://github.com/nexe/nexe/releases))
or you may want to customize what is built. See `nexe --help` for a list of options available when passing the [`--build`](#build-boolean) option. You will also need to ensure your environment is setup to [build node](https://github.com/nodejs/node/blob/master/BUILDING.md). Note: the `python` binary in your path should be an acceptable version of python 2. eg. Systems that have python2 will need to create a [symlink](https://github.com/nexe/nexe/issues/354#issuecomment-319874486).
### Linux and macOS
[Prerequisites & details](https://github.com/nodejs/node/blob/master/BUILDING.md#unix-and-macos)
### Windows
The fastest and most reliable way to get started is simply to run the commands below. If you'd rather read the details or perform a manual install of the prerequisites, [you can find that here](https://github.com/nodejs/node/blob/master/BUILDING.md#windows).
The instructions below are the fastest and most reliable method.
Run the following sets of commands with PowerShell (running as Administrator).
**Install all required build tools (and dependencies):**
```
Set-ExecutionPolicy Unrestricted -Force
iex ((New-Object System.Net.WebClient).DownloadString('https://boxstarter.org/bootstrapper.ps1'))
get-boxstarter -Force
Install-BoxstarterPackage https://raw.githubusercontent.com/nodejs/node/master/tools/bootstrap/windows_boxstarter -DisableReboots
```
**Set config:**
```
npm config set msvs_version 2019
npm config set python python3.8
```
Where `2019` is the version of Visual Studio you have (if you have it).
**Notes:**
- The above works and has been tested with node.js `14.5.4` and `15.8.0`
- Python 3 and Python 2 can coexist and `nexe` will still work, considering the `set config` area above
- Don't use `npm install windows-build-tools` unless you're having some type of issue, because the above commands configures and installs the latest/preferred too.
## Node.js API ## Node.js API
@@ -148,13 +117,11 @@ compile({
- default: true - default: true
- #### `build: boolean` - #### `build: boolean`
- Build node from source, passing this flag tells nexe to download and build from source. Subsequently using this flag will cause nexe to use the previously built binary. To rebuild, first add [`--clean`](#clean-boolean) - Build node from source, passing this flag tells nexe to download and build from source. Subsequently using this flag will cause nexe to use the previously built binary. To rebuild, first add [`--clean`](#clean-boolean)
- #### `remote: string`
- Provide a custom remote location for fetching pre-built nexe binaries from. This can either be an HTTP or HTTPS URL.
- default: `null`
- #### `asset: string` - #### `asset: string`
- Provide a pre-built nexe binary asset, this is a file path is resolved relative to cwd. - Provide a pre-built nexe binary asset, this can either be an http or https URL or a file path.
- default: `null`
- #### `python: string` - #### `python: string`
- On Linux this is the path pointing to your python3 executable - On Linux this is the path pointing to your python2 executable
- On Windows this is the directory where `python` can be accessed - On Windows this is the directory where `python` can be accessed
- default: `null` - default: `null`
- #### `flags: string[]` - #### `flags: string[]`
@@ -236,7 +203,7 @@ For examples, see the built in patches: [src/patches](src/patches).
- Quickly perform a replace in a file within the downloaded Node.js source. The rest arguments are passed along to `String.prototype.replace` - Quickly perform a replace in a file within the downloaded Node.js source. The rest arguments are passed along to `String.prototype.replace`
- `readFileAsync(filename: string): Promise<NexeFile>` - `readFileAsync(filename: string): Promise<NexeFile>`
- Access (or create) a file within the downloaded Node.js source. - Access (or create) a file within the downloaded Node.js source.
- `addResource(filename: string, contents: Buffer): Promise<void>` - `addResource(filename: string, contents: Buffer): void`
- Add a resource to the nexe bundle - Add a resource to the nexe bundle
- `files: NexeFile[]` - `files: NexeFile[]`
- The cache of the currently read, modified, or created files within the downloaded Node.js source. - The cache of the currently read, modified, or created files within the downloaded Node.js source.
@@ -250,20 +217,9 @@ Any modifications made to `NexeFile#contents` will be maintained in the cache _w
## Native Modules ## Native Modules
In order to use native modules, the native binaries must be shipped alongside the binary generated by nexe. TODO
## Troubleshooting - [ ] Implement support for `node-pre-gyp#find`.
`Error: Entry file "" not found!` means you need to provide `nexe` with input. Either use `-i` or pipe data to it.
`Error: https://github.com/nexe/nexe/releases/download/v3.3.3/windows-x64-15.8.0 is not available, create it using the --build flag` or similar message means that it either:
- You are having networking issues such as the download being blocked
- You should specify the target so `nexe` knows what version of the executable to use.
- See the [releases page](https://github.com/nexe/nexe/releases) to find the executable's version number
- Example
- `nexe -i "app.js" -r "public/**/*.html" -o "dist/myApp.exe" -t x64-14.15.3`
- where `-i` specifies the input, `-r` specifies resources to embed, `-o` specifies the output, `-t` specifies the target.
- Alternatively you can compile the executable yourself, see that section for details
## Contributing ## Contributing
@@ -271,10 +227,20 @@ Building
``` ```
$ git clone git@github.com:nexe/nexe.git $ git clone git@github.com:nexe/nexe.git
$ cd nexe $ cd nexe
$ npm i && npm run build $ yarn
``` ```
Testing Testing
``` ```
$ npm test $ npm test
``` ```
## Maintainers
[![Jared Allard](https://avatars.githubusercontent.com/u/2391349?s=130)](https://jaredallard.me/) | [![Caleb Boyd](https://avatars.githubusercontent.com/u/5818726?s=130)](https://github.com/calebboyd) | [![Dustin Greif](https://avatars.githubusercontent.com/u/3026298?s=130)](https://github.com/dgreif) |
---|---|---|
[Jared Allard](https://github.com/jaredallard) | [Caleb Boyd](http://github.com/calebboyd) | [Dustin Greif](https://github.com/dgreif) |
### Former
- [Craig Condon](http://crcn.codes/)
+7 -20
View File
@@ -19,13 +19,13 @@ schedules:
- master - master
jobs: jobs:
- job: BaseBuild - job: BaseBuild
timeoutInMinutes: 360 timeoutInMinutes: 180
strategy: strategy:
matrix: matrix:
linux: linux:
imageName: 'ubuntu-16.04' imageName: 'ubuntu-16.04'
mac: mac:
imageName: 'macos-10.15' imageName: 'macos-10.13'
windows_2017: windows_2017:
imageName: 'vs2017-win2016' imageName: 'vs2017-win2016'
windows_2015: windows_2015:
@@ -43,31 +43,18 @@ jobs:
- task: NodeTool@0 - task: NodeTool@0
displayName: Install Node displayName: Install Node
inputs: inputs:
versionSpec: '12.x' versionSpec: '10.x'
- script: sudo apt-get update && sudo apt-get install -y libc6-dev-i386 gcc-multilib g++-multilib - script: sudo apt-get update && sudo apt-get install -y libc6-dev-i386 gcc-multilib g++-multilib
condition: contains(variables['Agent.JobName'], 'linux') condition: contains(variables['Agent.JobName'], 'linux')
displayName: Install Linux build dependencies displayName: Install Linux build dependencies
- script: npm config set msvs_version 2015 - script: choco install visualcpp-build-tools --version 14.0.25420.1 -fy && npm config set msvs_version 2015
condition: contains(variables['Agent.JobName'], 'windows_2015') condition: contains(variables['Agent.JobName'], 'windows_2015')
displayName: Install Windows build dependencies displayName: Install Windows build dependencies
- script: choco install nasm -fy
displayName: Install Windows Assembler (NASM)
condition: contains(variables['Agent.JobName'], 'windows')
- script: npm install && npm run ci:build - script: npm install && npm run ci:build
displayName: Build Nexe displayName: Build Nexe
env: env:
GITHUB_TOKEN: $(PersonalGithubToken) GITHUB_TOKEN: $(PersonalGithubToken)
NEXE_VERBOSE: $(NEXE_VERBOSE) NEXE_VERBOSE: $(NEXE_VERBOSE)
msbuild_args: /3G
condition: not(contains(variables['Agent.JobName'], 'windows'))
- powershell: |
choco install nasm
npm install
npm run ci:build
displayName: Build Nexe (Windows)
condition: contains(variables['Agent.JobName'], 'windows')
env:
GITHUB_TOKEN: $(PersonalGithubToken)
NEXE_VERBOSE: $(NEXE_VERBOSE)
- script: npm run test:integration
displayName: Test built artefact
env:
NEXE_VERBOSE: $(NEXE_VERBOSE)
BUILD: ''
+19
View File
@@ -0,0 +1,19 @@
<!-- If you need help or think you have found a bug, please help us with your issue by entering the following information (otherwise you can delete this text): -->
**Is this a BUG or a FEATURE REQUEST?:**
**What happened:**
**What you expected to happen:**
**How to reproduce it (as minimally and precisely as possible):**
**Anything else we need to know?:**
**Environment**
- Platform(OS/Version):
- Host Node Version:
- Target Node Version:
- Nexe version:
- Python Version:
+2043 -7511
View File
File diff suppressed because it is too large Load Diff
+34 -50
View File
@@ -2,7 +2,7 @@
"name": "nexe", "name": "nexe",
"description": "Create a single executable out of your Node.js application", "description": "Create a single executable out of your Node.js application",
"license": "MIT", "license": "MIT",
"version": "5.0.0-beta.1", "version": "3.3.7",
"contributors": [ "contributors": [
"Craig Condon <craig.j.condon@gmail.com> (http://crcn.io)", "Craig Condon <craig.j.condon@gmail.com> (http://crcn.io)",
"Jared Allard <jaredallard@outlook.com>", "Jared Allard <jaredallard@outlook.com>",
@@ -11,14 +11,10 @@
"scripts": { "scripts": {
"ci:build": "ts-node tasks/build", "ci:build": "ts-node tasks/build",
"test": "mocha", "test": "mocha",
"test:integration": "node index.js --target 14.15.3 -i test/integration/index.js -o integration-tests --verbose -r test/integration -r node_modules/ && npm run test:integration:run",
"test:integration:run": "run-script-os",
"test:integration:run:win32": "integration-tests.exe",
"test:integration:run:default": "./integration-tests",
"lint": "tslint \"{src,plugins,tasks}/**/*.ts\" --fix", "lint": "tslint \"{src,plugins,tasks}/**/*.ts\" --fix",
"prepare": "npm run lint && npm run build && npm test", "prepare": "npm run lint && npm run build && npm test",
"prebuild": "rimraf lib", "prebuild": "rimraf lib",
"build": "tsc --declaration && tsc -p tasks && webpack", "build": "tsc --declaration",
"postbuild": "ts-node tasks/post-build" "postbuild": "ts-node tasks/post-build"
}, },
"repository": { "repository": {
@@ -34,59 +30,47 @@
"nexe": "index.js" "nexe": "index.js"
}, },
"engines": { "engines": {
"node": ">=10" "node": ">=6"
},
"mocha": {
"spec": "./test/**/*.spec.ts",
"checkLeaks": true,
"require": [
"ts-node/register"
]
}, },
"dependencies": { "dependencies": {
"@calebboyd/semaphore": "^1.3.1", "@calebboyd/semaphore": "^1.3.1",
"@yarnpkg/fslib": "^3.0.0-rc.43", "app-builder": "^5.2.0",
"@yarnpkg/libzip": "^3.0.0-rc.43",
"app-builder": "^7.0.4",
"archiver": "^5.3.1",
"caw": "^2.0.1", "caw": "^2.0.1",
"chalk": "^2.4.2", "chalk": "^2.4.2",
"download": "^8.0.0", "cherow": "1.6.9",
"globby": "^11.0.2", "download": "^7.1.0",
"got": "^12.6.0", "globby": "^9.2.0",
"meriyah": "^4.3.5", "got": "^9.6.0",
"minimist": "^1.2.8", "minimist": "^1.2.0",
"mkdirp": "^1.0.4", "mkdirp": "^0.5.1",
"multistream": "^4.1.0", "multistream": "^2.1.1",
"ora": "^3.4.0", "ora": "^3.4.0",
"resolve-dependencies": "^6.0.9", "pify": "^4.0.1",
"rimraf": "^3.0.2", "resolve-dependencies": "^4.2.5",
"run-script-os": "^1.1.6", "rimraf": "^2.6.3"
"webpack-config-prefabs": "0.0.5"
}, },
"devDependencies": { "devDependencies": {
"@types/archiver": "^5.3.2", "@types/chai": "^4.1.7",
"@types/chai": "^4", "@types/download": "^6.2.4",
"@types/download": "^8", "@types/execa": "^0.9.0",
"@types/globby": "^9", "@types/globby": "^9.1.0",
"@types/lodash": "^4.14.192", "@types/got": "^9.6.0",
"@types/minimist": "^1.2.2", "@types/minimist": "^1.2.0",
"@types/mkdirp": "^1.0.2", "@types/mkdirp": "^0.5.2",
"@types/mocha": "^10.0.1", "@types/mocha": "^5.2.7",
"@types/multistream": "^4.1.0", "@types/multistream": "^2.1.1",
"@types/ora": "^3.2.0", "@types/ora": "^3.2.0",
"@types/rimraf": "3.0.2", "@types/pify": "3.0.2",
"@types/semver": "^7.3.13", "@types/rimraf": "2.0.2",
"chai": "^4.3.7", "@types/semver": "^6.0.0",
"execa": "^5.1.1", "chai": "^4.2.0",
"lodash": "^4.17.21", "execa": "^1.0.0",
"mocha": "^10.2.0", "mocha": "^6.1.4",
"prettier": "^2.8.7", "prettier": "^1.18.2",
"ts-node": "^10.9.1", "ts-node": "^8.3.0",
"tslint": "^6.1.1", "tslint": "^5.17.0",
"tslint-config-prettier": "^1.18.0", "tslint-config-prettier": "^1.18.0",
"tslint-plugin-prettier": "^2.3.0", "tslint-plugin-prettier": "^2.0.1",
"typescript": "^5.0.3", "typescript": "^3.5.2"
"webpack-cli": "^5.0.1"
} }
} }
+43 -52
View File
@@ -11,16 +11,12 @@ import {
dequote, dequote,
isWindows, isWindows,
bound, bound,
semverGt, semverGt
wrap,
} from './util' } from './util'
import { NexeOptions, version } from './options' import { NexeOptions, version } from './options'
import { NexeTarget } from './target' import { NexeTarget } from './target'
import { PassThrough, Readable, Stream, Transform } from 'stream' import combineStreams = require('multistream')
import MultiStream = require('multistream')
import { Bundle, toStream } from './fs/bundle' import { Bundle, toStream } from './fs/bundle'
import { File } from 'resolve-dependencies'
import { FactoryStream, LazyStream } from 'multistream'
const isBsd = Boolean(~process.platform.indexOf('bsd')) const isBsd = Boolean(~process.platform.indexOf('bsd'))
const make = isWindows ? 'vcbuild.bat' : isBsd ? 'gmake' : 'make' const make = isWindows ? 'vcbuild.bat' : isBsd ? 'gmake' : 'make'
@@ -31,7 +27,7 @@ type StringReplacer = (match: string) => string
export interface NexeFile { export interface NexeFile {
filename: string filename: string
absPath: string absPath: string
contents: string | Buffer contents: string
} }
export { NexeOptions } export { NexeOptions }
@@ -112,10 +108,12 @@ export class NexeCompiler {
//SOMEDAY iterate over multiple targets with `--outDir` //SOMEDAY iterate over multiple targets with `--outDir`
this.targets = options.targets as NexeTarget[] this.targets = options.targets as NexeTarget[]
this.target = this.targets[0] this.target = this.targets[0]
if (!/https?\:\/\//.test(options.remote)) { if (options.asset && options.asset.startsWith('http')) {
throw new NexeError(`Invalid remote URI scheme (must be http or https): ${options.remote}`) this.remoteAsset = options.asset
} else {
this.remoteAsset =
'https://github.com/nexe/nexe/releases/download/v3.3.3/' + this.target.toString()
} }
this.remoteAsset = options.remote + this.target.toString()
this.src = join(this.options.temp, this.target.version) this.src = join(this.options.temp, this.target.version)
this.configureScript = configure + (semverGt(this.target.version, '10.10.0') ? '.py' : '') this.configureScript = configure + (semverGt(this.target.version, '10.10.0') ? '.py' : '')
this.nodeSrcBinPath = isWindows this.nodeSrcBinPath = isWindows
@@ -138,23 +136,31 @@ export class NexeCompiler {
} }
@bound @bound
addResource(absoluteFileName: string, content?: Buffer | string | File) { addResource(file: string, content?: Buffer | string) {
return this.bundle.addResource(absoluteFileName, content) return this.bundle.addResource(file, content)
}
get binaryConfiguration() {
return { resources: this.bundle.index }
}
get resourceSize() {
return this.bundle.blobSize
} }
@bound @bound
async readFileAsync(file: string) { async readFileAsync(file: string) {
this.assertBuild() this.assertBuild()
let cachedFile = this.files.find((x) => normalize(x.filename) === normalize(file)) let cachedFile = this.files.find(x => normalize(x.filename) === normalize(file))
if (!cachedFile) { if (!cachedFile) {
const absPath = join(this.src, file) const absPath = join(this.src, file)
cachedFile = { cachedFile = {
absPath, absPath,
filename: file, filename: file,
contents: await readFileAsync(absPath, 'utf-8').catch((x) => { contents: await readFileAsync(absPath, 'utf-8').catch(x => {
if (x.code === 'ENOENT') return '' if (x.code === 'ENOENT') return ''
throw x throw x
}), })
} }
this.files.push(cachedFile) this.files.push(cachedFile)
} }
@@ -170,11 +176,11 @@ export class NexeCompiler {
@bound @bound
async replaceInFileAsync(file: string, replace: string | RegExp, value: string | StringReplacer) { async replaceInFileAsync(file: string, replace: string | RegExp, value: string | StringReplacer) {
const entry = await this.readFileAsync(file) const entry = await this.readFileAsync(file)
entry.contents = entry.contents.toString().replace(replace, value as any) entry.contents = entry.contents.replace(replace, value as any)
} }
@bound @bound
async setFileContentsAsync(file: string, contents: string | Buffer) { async setFileContentsAsync(file: string, contents: string) {
const entry = await this.readFileAsync(file) const entry = await this.readFileAsync(file)
entry.contents = contents entry.contents = contents
} }
@@ -192,7 +198,7 @@ export class NexeCompiler {
} }
public getNodeExecutableLocation(target?: NexeTarget) { public getNodeExecutableLocation(target?: NexeTarget) {
if (this.options.asset) { if (this.options.asset && !this.options.asset.startsWith('http')) {
return resolve(this.options.cwd, this.options.asset) return resolve(this.options.cwd, this.options.asset)
} }
if (target) { if (target) {
@@ -205,11 +211,11 @@ export class NexeCompiler {
if (this.log.verbose) { if (this.log.verbose) {
this.compileStep!.pause() this.compileStep!.pause()
} }
return new Promise<void>((resolve, reject) => { return new Promise((resolve, reject) => {
spawn(command, args, { spawn(command, args, {
cwd: this.src, cwd: this.src,
env: this.env, env: this.env,
stdio: this.log.verbose ? 'inherit' : 'ignore', stdio: this.log.verbose ? 'inherit' : 'ignore'
}) })
.once('error', (e: Error) => { .once('error', (e: Error) => {
if (this.log.verbose) { if (this.log.verbose) {
@@ -236,7 +242,7 @@ export class NexeCompiler {
} }
return this._runBuildCommandAsync(this.env.PYTHON || 'python', [ return this._runBuildCommandAsync(this.env.PYTHON || 'python', [
this.configureScript, this.configureScript,
...this.options.configure, ...this.options.configure
]) ])
} }
@@ -255,7 +261,10 @@ export class NexeCompiler {
return createReadStream(this.getNodeExecutableLocation()) return createReadStream(this.getNodeExecutableLocation())
} }
private async _shouldCompileBinaryAsync(binary: NodeJS.ReadableStream | null, location: string) { private async _shouldCompileBinaryAsync(
binary: NodeJS.ReadableStream | null,
location: string | undefined
) {
//SOMEDAY combine make/configure/vcBuild/and modified times of included files //SOMEDAY combine make/configure/vcBuild/and modified times of included files
const { snapshot, build } = this.options const { snapshot, build } = this.options
@@ -289,40 +298,22 @@ export class NexeCompiler {
return [this.shims.join(''), this.startup].join(';') return [this.shims.join(''), this.startup].join(';')
} }
private async _assembleDeliverable(binary: Readable) { private _assembleDeliverable(binary: NodeJS.ReadableStream) {
if (!this.options.mangle) { if (!this.options.mangle) {
return binary return binary
} }
const launchCode = this.code()
const codeSize = Buffer.byteLength(launchCode)
const sentinel = Buffer.from('<nexe~~sentinel>')
let vfsSize = 0 const startup = this.code(),
const streams = [ codeSize = Buffer.byteLength(startup)
const lengths = Buffer.from(Array(16))
lengths.writeDoubleLE(codeSize, 0)
lengths.writeDoubleLE(this.bundle.blobSize, 8)
return combineStreams([
binary, binary,
toStream(launchCode), toStream(startup),
this.bundle.toStream().pipe( this.bundle.toStream(),
new Transform({ toStream(Buffer.concat([Buffer.from('<nexe~~sentinel>'), lengths]))
transform: (chunk, _, cb) => { ])
vfsSize || this.bundle.finalize()
chunk && (vfsSize += chunk.length)
cb(null, chunk)
},
})
),
]
let done = false
return new MultiStream((cb) => {
if (done) cb(null, null)
else if (streams.length) cb(null, streams.shift() as Readable)
else {
done = true
const trailers = Buffer.alloc(16)
trailers.writeDoubleLE(codeSize, 0)
trailers.writeDoubleLE(vfsSize, 8)
cb(null, toStream(Buffer.concat([sentinel, trailers])))
}
})
} }
} }
-395
View File
@@ -1,395 +0,0 @@
import { Libzip } from '@yarnpkg/libzip'
import {
FakeFS,
PortablePath,
RmdirOptions,
OpendirOptions,
BasePortableFakeFS,
Dirent,
Dir,
MkdirOptions,
FSPath,
ppath,
npath,
Filename,
CustomDir,
} from '@yarnpkg/fslib'
import { ZipFS, ZipOpenFS } from '@yarnpkg/libzip'
import { resolve, toNamespacedPath } from 'path'
import { constants } from 'fs'
export type SnapshotZipFSOptions = {
baseFs: FakeFS<PortablePath>
libzip: Libzip | (() => Libzip)
zipFs: ZipFS
root: string
}
const uniqBy = (arr: Array<string | Dirent>, pick: (...arg: any[]) => any) => {
const seen = new Set()
return arr.filter((x) => {
const key = pick(x)
if (seen.has(key)) return false
seen.add(key)
return true
})
}
function uniqReaddir(arr: Array<string | Dirent>) {
return uniqBy(arr, (s: string | Dirent) => (typeof s === 'string' ? s : s.name))
}
export class SnapshotZipFS extends BasePortableFakeFS {
zipFs: ZipFS
baseFs: FakeFS<PortablePath>
root: string
magic: number
constructor(opts: SnapshotZipFSOptions) {
super()
this.zipFs = opts.zipFs
this.baseFs = opts.baseFs
this.root = opts.root
this.magic = 0x2a << 24
}
private readonly fdMap: Map<number, [ZipFS, number]> = new Map()
private nextFd = 3
async makeCallPromise<T>(
p: FSPath<PortablePath>,
discard: () => Promise<T>,
accept: (zipFS: ZipFS, zipInfo: { subPath: PortablePath }) => Promise<T>,
{ requireSubpath = true }: { requireSubpath?: boolean } = {}
): Promise<T> {
if (typeof p !== 'string') return await discard()
const normalizedP = this.resolve(p)
const zipInfo = this.findZip(normalizedP)
if (!zipInfo) return await discard()
if (requireSubpath && zipInfo.subPath === '/') return await discard()
return await accept(this.zipFs, zipInfo)
}
makeCallSync<T>(
p: FSPath<PortablePath>,
discard: () => T,
accept: (zipFS: ZipFS, zipInfo: { subPath: PortablePath; archivePath: string }) => T,
{ requireSubpath = true }: { requireSubpath?: boolean } = {}
): T {
if (typeof p !== 'string') return discard()
const normalizedP = this.resolve(p)
const zipInfo = this.findZip(normalizedP)
if (!zipInfo) return discard()
if (requireSubpath && zipInfo.subPath === '/') return discard()
return accept(this.zipFs, { archivePath: '', ...zipInfo })
}
async realpathPromise(p: PortablePath) {
return await this.realpathSync(p)
}
realpathSync(p: PortablePath) {
return this.makeCallSync(
p,
() => this.baseFs.realpathSync(p),
(zipFs, { subPath }) => {
if (zipFs.lstatSync(subPath).isSymbolicLink()) {
return zipFs.realpathSync(subPath)
} else {
// return the original path in case it wasn't under /snapshot, e.g. if it was for a node module - otherwise the node module parent path is the wrong one (and other things resolve relative to that)
return p
}
}
)
}
findZip(p: PortablePath) {
p = this.resolve(p)
const snapshotPP = npath.toPortablePath('/snapshot')
const pathsToTry = Array.from(
new Set([
p,
resolve('/snapshot', p),
ppath.resolve(
snapshotPP,
npath.toPortablePath(
npath.relative(npath.fromPortablePath(this.root), npath.fromPortablePath(p))
)
),
ppath.resolve(
snapshotPP,
npath.toPortablePath(
npath.relative(
toNamespacedPath(npath.fromPortablePath(this.root)),
toNamespacedPath(npath.fromPortablePath(p))
)
)
),
])
)
for (const path of pathsToTry) {
const portablePath = npath.toPortablePath(path)
if (this.zipFs.existsSync(portablePath)) {
return {
subPath: portablePath,
}
}
}
}
async copyFilePromise(sourceP: PortablePath, destP: PortablePath, flags: number = 0) {
const fallback = async (
sourceFs: FakeFS<PortablePath>,
sourceP: PortablePath,
destFs: FakeFS<PortablePath>,
destP: PortablePath
) => {
if ((flags & constants.COPYFILE_FICLONE_FORCE) !== 0)
throw Object.assign(
new Error(`EXDEV: cross-device clone not permitted, copyfile '${sourceP}' -> ${destP}'`),
{ code: `EXDEV` }
)
if (flags & constants.COPYFILE_EXCL && (await this.existsPromise(sourceP)))
throw Object.assign(
new Error(`EEXIST: file already exists, copyfile '${sourceP}' -> '${destP}'`),
{ code: `EEXIST` }
)
let content
try {
content = await sourceFs.readFilePromise(sourceP)
} catch (error) {
throw Object.assign(
new Error(`EINVAL: invalid argument, copyfile '${sourceP}' -> '${destP}'`),
{ code: `EINVAL` }
)
}
await destFs.writeFilePromise(destP, content)
}
return await this.makeCallPromise(
sourceP,
async () => {
return await this.baseFs.copyFilePromise(sourceP, destP, flags)
},
async (zipFsS, { subPath: subPathS }) => {
return await fallback(zipFsS, subPathS, this.baseFs, destP)
}
)
}
copyFileSync(sourceP: PortablePath, destP: PortablePath, flags: number = 0) {
const fallback = (
sourceFs: FakeFS<PortablePath>,
sourceP: PortablePath,
destFs: FakeFS<PortablePath>,
destP: PortablePath
) => {
if ((flags & constants.COPYFILE_FICLONE_FORCE) !== 0)
throw Object.assign(
new Error(`EXDEV: cross-device clone not permitted, copyfile '${sourceP}' -> ${destP}'`),
{ code: `EXDEV` }
)
if (flags & constants.COPYFILE_EXCL && this.existsSync(sourceP))
throw Object.assign(
new Error(`EEXIST: file already exists, copyfile '${sourceP}' -> '${destP}'`),
{ code: `EEXIST` }
)
let content
try {
content = sourceFs.readFileSync(sourceP)
} catch (error) {
throw Object.assign(
new Error(`EINVAL: invalid argument, copyfile '${sourceP}' -> '${destP}'`),
{ code: `EINVAL` }
)
}
destFs.writeFileSync(destP, content)
}
return this.makeCallSync(
sourceP,
() => {
return this.baseFs.copyFileSync(sourceP, destP, flags)
},
(zipFsS, { subPath: subPathS }) => {
return fallback(zipFsS, subPathS, this.baseFs, destP)
}
)
}
async readdirPromise(p: PortablePath): Promise<Array<Filename>>
async readdirPromise(
p: PortablePath,
opts: { withFileTypes: false } | null
): Promise<Array<Filename>>
async readdirPromise(p: PortablePath, opts: { withFileTypes: true }): Promise<Array<Dirent>>
async readdirPromise(
p: PortablePath,
opts: { withFileTypes: boolean }
): Promise<Array<Filename> | Array<Dirent>>
async readdirPromise(
p: PortablePath,
opts?: { withFileTypes?: boolean } | null
): Promise<Array<string | Dirent>> {
const fallback = async () => {
return await this.baseFs.readdirPromise(p, opts as any)
}
return await this.makeCallPromise(
p,
fallback,
async (zipFs, { subPath }) => {
const fallbackPaths: Array<string | Dirent> = await fallback().catch(() => [])
return Promise.resolve(
uniqReaddir(fallbackPaths.concat(await zipFs.readdirPromise(subPath, opts as any)))
)
},
{
requireSubpath: false,
}
)
}
readdirSync(p: PortablePath): Array<Filename>
readdirSync(p: PortablePath, opts: { withFileTypes: false } | null): Array<Filename>
readdirSync(p: PortablePath, opts: { withFileTypes: true }): Array<Dirent>
readdirSync(p: PortablePath, opts: { withFileTypes: boolean }): Array<Filename> | Array<Dirent>
readdirSync(p: PortablePath, opts?: { withFileTypes?: boolean } | null): Array<string | Dirent> {
const fallback = () => {
return this.baseFs.readdirSync(p, opts as any)
}
return this.makeCallSync(
p,
fallback,
(zipFs, { subPath }) => {
let fallbackPaths: Array<string | Dirent> = []
try {
fallbackPaths = fallback()
} catch (e) {}
return fallbackPaths.concat(uniqReaddir(zipFs.readdirSync(subPath, opts as any)))
},
{
requireSubpath: false,
}
)
}
async mkdirPromise(p: PortablePath, opts?: MkdirOptions) {
return await this.baseFs.mkdirPromise(p, opts)
}
mkdirSync(p: PortablePath, opts?: MkdirOptions) {
return this.baseFs.mkdirSync(p, opts)
}
async rmdirPromise(p: PortablePath, opts?: RmdirOptions) {
return await this.baseFs.rmdirPromise(p, opts)
}
rmdirSync(p: PortablePath, opts?: RmdirOptions) {
return this.baseFs.rmdirSync(p, opts)
}
async opendirPromise(p: PortablePath, opts?: OpendirOptions) {
return this.opendirSync(p, opts)
}
opendirSync(p: PortablePath, opts?: OpendirOptions) {
const zipInfo = this.findZip(p)
let zipFsDir: Dir<PortablePath> | null = null
if (zipInfo) {
zipFsDir = this.zipFs.opendirSync(zipInfo.subPath)
}
let realFsDir: Dir<PortablePath> | null = null
try {
realFsDir = this.baseFs.opendirSync(p)
} catch (e) {
if (!zipFsDir) throw e
}
const seen = new Set()
const nextDirent = () => {
const entry = realFsDir?.readSync() || zipFsDir?.readSync()
if (entry && !seen.has(entry.name)) {
seen.add(entry.name)
return entry
}
return null
}
const onClose = () => {
zipFsDir?.closeSync()
realFsDir?.closeSync()
}
return new CustomDir(p, nextDirent, { onClose })
}
accessPromise = ZipOpenFS.prototype.accessPromise
accessSync = ZipOpenFS.prototype.accessSync
appendFilePromise = ZipOpenFS.prototype.appendFilePromise
appendFileSync = ZipOpenFS.prototype.appendFileSync
chmodPromise = ZipOpenFS.prototype.chmodPromise
chmodSync = ZipOpenFS.prototype.chmodSync
fchmodPromise = ZipOpenFS.prototype.fchmodPromise
fchmodSync = ZipOpenFS.prototype.fchmodSync
chownPromise = ZipOpenFS.prototype.chownPromise
chownSync = ZipOpenFS.prototype.chownSync
fchownPromise = ZipOpenFS.prototype.fchownPromise
fchownSync = ZipOpenFS.prototype.fchownSync
closePromise = ZipOpenFS.prototype.closePromise
closeSync = ZipOpenFS.prototype.closeSync
createReadStream = ZipOpenFS.prototype.createReadStream
createWriteStream = ZipOpenFS.prototype.createWriteStream
existsPromise = ZipOpenFS.prototype.existsPromise
existsSync = ZipOpenFS.prototype.existsSync
fstatPromise = ZipOpenFS.prototype.fstatPromise
fstatSync = ZipOpenFS.prototype.fstatSync
getExtractHint = ZipOpenFS.prototype.getExtractHint
getRealPath = ZipOpenFS.prototype.getRealPath
linkPromise = ZipOpenFS.prototype.linkPromise
linkSync = ZipOpenFS.prototype.linkSync
lstatPromise = ZipOpenFS.prototype.lstatPromise
lstatSync = ZipOpenFS.prototype.lstatSync
openPromise = ZipOpenFS.prototype.openPromise
openSync = ZipOpenFS.prototype.openSync
readFilePromise = ZipOpenFS.prototype.readFilePromise
readFileSync = ZipOpenFS.prototype.readFileSync
readlinkPromise = ZipOpenFS.prototype.readlinkPromise
readlinkSync = ZipOpenFS.prototype.readlinkSync
readPromise = ZipOpenFS.prototype.readPromise
readSync = ZipOpenFS.prototype.readSync
renamePromise = ZipOpenFS.prototype.renamePromise
renameSync = ZipOpenFS.prototype.renameSync
resolve = ZipOpenFS.prototype.resolve
statPromise = ZipOpenFS.prototype.statPromise
statSync = ZipOpenFS.prototype.statSync
symlinkPromise = ZipOpenFS.prototype.symlinkPromise
symlinkSync = ZipOpenFS.prototype.symlinkSync
truncatePromise = ZipOpenFS.prototype.truncatePromise
truncateSync = ZipOpenFS.prototype.truncateSync
ftruncatePromise = ZipOpenFS.prototype.ftruncatePromise
ftruncateSync = ZipOpenFS.prototype.ftruncateSync
unlinkPromise = ZipOpenFS.prototype.unlinkPromise
unlinkSync = ZipOpenFS.prototype.unlinkSync
unwatchFile = ZipOpenFS.prototype.unwatchFile
utimesPromise = ZipOpenFS.prototype.utimesPromise
utimesSync = ZipOpenFS.prototype.utimesSync
lutimesPromise = ZipOpenFS.prototype.utimesPromise
lutimesSync = ZipOpenFS.prototype.utimesSync
watch = ZipOpenFS.prototype.watch
watchFile = ZipOpenFS.prototype.watchFile
writeFilePromise = ZipOpenFS.prototype.writeFilePromise
writeFileSync = ZipOpenFS.prototype.writeFileSync
writePromise = ZipOpenFS.prototype.writePromise
writeSync = ZipOpenFS.prototype.writeSync
// @ts-ignore
remapFd = ZipOpenFS.prototype.remapFd
}
+50 -31
View File
@@ -1,51 +1,70 @@
import { stat as getStat, Stats, createReadStream } from 'fs'
import { relative } from 'path' import { relative } from 'path'
import { Readable } from 'stream' import { Readable } from 'stream'
import { File } from 'resolve-dependencies' import combineStreams = require('multistream')
import archiver from 'archiver'
function makeRelativeToZip(cwd: string, path: string) { const stat = (file: string): Promise<Stats> => {
return '/snapshot/' + relative(cwd, path) return new Promise((resolve, reject) => {
getStat(file, (err, stats) => (err ? reject(err) : resolve(stats)))
})
}
function makeRelative(cwd: string, path: string) {
return './' + relative(cwd, path)
} }
export function toStream(content: Buffer | string) { export function toStream(content: Buffer | string) {
const readable = new Readable({ const readable = new Readable({ read() {} })
read() { readable.push(content)
this.push(content) readable.push(null)
this.push(null)
},
})
return readable return readable
} }
export class Bundle {
export type File = { absPath: string; contents: string; deps: FileMap }
export type FileMap = { [absPath: string]: File | null }
export interface BundleOptions {
entries: string[]
cwd: string cwd: string
files = new Set<string>() expand: boolean
zip: any loadContent: boolean
files: FileMap
}
export class Bundle {
constructor({ cwd }: { cwd: string } = { cwd: process.cwd() }) { constructor({ cwd }: { cwd: string } = { cwd: process.cwd() }) {
this.cwd = cwd this.cwd = cwd
this.zip = archiver('zip')
} }
cwd: string
blobSize: number = 0
index: { [relativeFilePath: string]: [number, number] } = {}
streams: (Readable | (() => Readable))[] = []
get list() { async addResource(absoluteFileName: string, content?: Buffer | string) {
return Array.from(this.files) let length = 0
} if (content !== undefined) {
length = Buffer.byteLength(content)
public addResource(absoluteFileName: string, content?: File | Buffer | string) { } else {
const destPath = makeRelativeToZip(this.cwd, absoluteFileName) const stats = await stat(absoluteFileName)
if (!this.files.has(destPath)) { length = stats.size
if (content == null) {
this.zip.file(absoluteFileName, { name: destPath })
} else {
this.zip.append(content, { name: destPath })
}
this.files.add(destPath)
} }
const start = this.blobSize
this.blobSize += length
this.index[makeRelative(this.cwd, absoluteFileName)] = [start, length]
this.streams.push(() => (content ? toStream(content) : createReadStream(absoluteFileName)))
} }
public finalize() { concat() {
return this.zip.finalize() throw new Error('Not Implemented')
} }
public toStream(): Readable { toStream() {
return this.zip return combineStreams(this.streams)
}
toJSON() {
return this.index
} }
} }
+340 -78
View File
@@ -1,108 +1,370 @@
import { ZipFS, getLibzipSync } from '@yarnpkg/libzip' import { Stats } from 'fs'
import { patchFs, npath, PosixFS, NodeFS } from '@yarnpkg/fslib'
import { SnapshotZipFS } from './SnapshotZipFS'
import * as assert from 'assert'
import * as constants from 'constants'
import { dirname } from 'path'
export interface NexeHeader { export interface NexeBinary {
blobPath: string blobPath: string
resources: { [key: string]: number[] }
layout: { layout: {
stat: Stats
resourceStart: number resourceStart: number
resourceSize: number contentSize?: number
contentSize: number contentStart?: number
contentStart: number resourceSize?: number
} }
} }
let originalFsMethods: any = null let originalFsMethods: any = null
let lazyRestoreFs = () => {} let lazyRestoreFs = () => {}
const patches = (process as any).nexe.patches || {}
const originalPatches = { ...patches }
delete (process as any).nexe
function shimFs(binary: NexeHeader, fs: typeof import('fs') = require('fs')) { function shimFs(binary: NexeBinary, fs: any = require('fs')) {
if (originalFsMethods !== null) { if (originalFsMethods !== null) {
return return
} }
originalFsMethods = Object.assign({}, fs) originalFsMethods = Object.assign({}, fs)
const { blobPath, resources: manifest } = binary,
const realFs: typeof fs = { ...fs } { resourceStart, stat } = binary.layout,
const nodeFs = new NodeFS(realFs) directories: { [key: string]: { [key: string]: boolean } } = {},
notAFile = '!@#$%^&*',
const blob = Buffer.allocUnsafe(binary.layout.resourceSize) isWin = process.platform.startsWith('win'),
const blobFd = realFs.openSync(binary.blobPath, 'r') isString = (x: any): x is string => typeof x === 'string' || x instanceof String,
const bytesRead = realFs.readSync( noop = () => {},
blobFd, path = require('path'),
blob, baseDir = path.dirname(process.execPath)
0,
binary.layout.resourceSize,
binary.layout.resourceStart
)
assert.equal(bytesRead, binary.layout.resourceSize)
const zipFs = new ZipFS(blob, { readOnly: true })
const snapshotZipFS = new SnapshotZipFS({
libzip: getLibzipSync(),
zipFs,
baseFs: nodeFs,
root: dirname(process.argv[0]),
})
const posixSnapshotZipFs = new PosixFS(snapshotZipFS)
patchFs(fs, posixSnapshotZipFs)
let log = (_: string) => true let log = (_: string) => true
if ((process.env.DEBUG || '').toLowerCase().includes('nexe:require')) { if ((process.env.DEBUG || '').toLowerCase().includes('nexe:require')) {
process.stderr.write( log = (text: string) => process.stderr.write('[nexe] - ' + text + '\n')
// @ts-ignore }
`[nexe] - FILES ${JSON.stringify(Array.from(zipFs.entries.keys()), null, 4)}\n`
) const winPath = (key: string) => {
process.stderr.write( if (isWin && key.substr(1, 2) === ':\\') {
// @ts-ignore key = key[0].toUpperCase() + key.substr(1)
`[nexe] - DIRECTORIES ${JSON.stringify(Array.from(zipFs.listings.keys()), null, 4)}\n`
)
log = (text: string) => {
return process.stderr.write(`[nexe] - ${text}\n`)
} }
return key
} }
function internalModuleReadFile(this: any, original: any, ...args: any[]) {
log(`internalModuleReadFile ${args[0]}`) const getKey = function getKey(filepath: string | Buffer | null): string {
try { if (Buffer.isBuffer(filepath)) {
return posixSnapshotZipFs.readFileSync(args[0], 'utf-8') filepath = filepath.toString()
} catch (e) {
return ''
} }
if (!isString(filepath)) {
return notAFile
}
let key = path.resolve(baseDir, filepath)
return winPath(key)
} }
if (patches.internalModuleReadFile) {
patches.internalModuleReadFile = internalModuleReadFile const statTime = function() {
} return {
let returningArray: boolean dev: 0,
patches.internalModuleReadJSON = function (this: any, original: any, ...args: any[]) { ino: 0,
if (returningArray == null) returningArray = Array.isArray(original.call(this, '')) nlink: 0,
const res = internalModuleReadFile.call(this, original, ...args) rdev: 0,
return returningArray && !Array.isArray(res) uid: 123,
? res === '' gid: 500,
? [] blksize: 4096,
: [res, /"(main|name|type|exports|imports)"/.test(res)] blocks: 0,
: res atime: new Date(stat.atime),
} atimeMs: stat.atime.getTime(),
patches.internalModuleStat = function (this: any, original: any, ...args: any[]) { mtime: new Date(stat.mtime),
log(`internalModuleStat ${args[0]}`) mtimeMs: stat.mtime.getTime(),
try { ctime: new Date(stat.ctime),
const stat = posixSnapshotZipFs.statSync(args[0]) ctimMs: stat.ctime.getTime(),
if (stat.isDirectory()) return 1 birthtime: new Date(stat.birthtime),
return 0 birthtimeMs: stat.birthtime.getTime()
} catch (e) {
return -constants.ENOENT
} }
} }
const createStat = function(extensions: any) {
return Object.assign(new fs.Stats(), binary.layout.stat, statTime(), extensions)
}
const ownStat = function(filepath: any) {
setupManifest()
const key = getKey(filepath)
if (directories[key]) {
let mode = binary.layout.stat.mode
mode |= fs.constants.S_IFDIR
mode &= ~fs.constants.S_IFREG
return createStat({ mode, size: 0 })
}
if (manifest[key]) {
return createStat({ size: manifest[key][1] })
}
}
function makeLong(filepath: string) {
return (path as any)._makeLong && (path as any)._makeLong(filepath)
}
function fileOpts(options: any) {
return !options ? {} : isString(options) ? { encoding: options } : options
}
let setupManifest = () => {
Object.keys(manifest).forEach(filepath => {
const entry = manifest[filepath]
const absolutePath = getKey(filepath)
const longPath = makeLong(absolutePath)
const normalizedPath = winPath(path.normalize(filepath))
if (!manifest[absolutePath]) {
manifest[absolutePath] = entry
}
if (longPath && !manifest[longPath]) {
manifest[longPath] = entry
}
if (!manifest[normalizedPath]) {
manifest[normalizedPath] = manifest[filepath]
}
let currentDir = path.dirname(absolutePath)
let prevDir = absolutePath
while (currentDir !== prevDir) {
directories[currentDir] = directories[currentDir] || {}
directories[currentDir][path.basename(prevDir)] = true
const longDir = makeLong(currentDir)
if (longDir && !directories[longDir]) {
directories[longDir] = directories[currentDir]
}
prevDir = currentDir
currentDir = path.dirname(currentDir)
}
})
;(manifest[notAFile] as any) = false
;(directories[notAFile] as any) = false
setupManifest = noop
}
//naive patches intended to work for most use cases
const nfs: any = {
existsSync: function existsSync(filepath: string) {
setupManifest()
const key = getKey(filepath)
if (manifest[key] || directories[key]) {
return true
}
return originalFsMethods.existsSync.apply(fs, arguments)
},
realpath: function realpath(filepath: any, options: any, cb: any): void {
setupManifest()
const key = getKey(filepath)
if (isString(filepath) && (manifest[filepath] || manifest[key])) {
return process.nextTick(() => cb(null, filepath))
}
return originalFsMethods.realpath.call(fs, filepath, options, cb)
},
realpathSync: function realpathSync(filepath: any, options: any) {
setupManifest()
const key = getKey(filepath)
if (manifest[key]) {
return filepath
}
return originalFsMethods.realpathSync.call(fs, filepath, options)
},
readdir: function readdir(filepath: string | Buffer, options: any, callback: any) {
setupManifest()
const dir = directories[getKey(filepath)]
if (dir) {
if ('function' === typeof options) {
callback = options
options = { encoding: 'utf8' }
}
process.nextTick(() => callback(null, Object.keys(dir)))
} else {
return originalFsMethods.readdir.apply(fs, arguments)
}
},
readdirSync: function readdirSync(filepath: string | Buffer, options: any) {
setupManifest()
const dir = directories[getKey(filepath)]
if (dir) {
return Object.keys(dir)
}
return originalFsMethods.readdirSync.apply(fs, arguments)
},
readFile: function readFile(filepath: any, options: any, callback: any) {
setupManifest()
const entry = manifest[getKey(filepath)]
if (!entry) {
return originalFsMethods.readFile.apply(fs, arguments)
}
const [offset, length] = entry
const resourceOffset = resourceStart + offset
const encoding = fileOpts(options).encoding
callback = typeof options === 'function' ? options : callback
originalFsMethods.open(blobPath, 'r', function(err: Error, fd: number) {
if (err) return callback(err, null)
originalFsMethods.read(fd, Buffer.alloc(length), 0, length, resourceOffset, function(
error: Error,
bytesRead: number,
result: Buffer
) {
if (error) {
return originalFsMethods.close(fd, function() {
callback(error, null)
})
}
originalFsMethods.close(fd, function(err: Error) {
if (err) {
return callback(err, result)
}
callback(err, encoding ? result.toString(encoding) : result)
})
})
})
},
createReadStream: function createReadStream(filepath: any, options: any) {
setupManifest()
const entry = manifest[getKey(filepath)]
if (!entry) {
return originalFsMethods.createReadStream.apply(fs, arguments)
}
const [offset, length] = entry
const resourceOffset = resourceStart + offset
const opts = fileOpts(options)
return originalFsMethods.createReadStream(
blobPath,
Object.assign({}, opts, {
start: resourceOffset,
end: resourceOffset + length - 1
})
)
},
readFileSync: function readFileSync(filepath: any, options: any) {
setupManifest()
const entry = manifest[getKey(filepath)]
if (!entry) {
return originalFsMethods.readFileSync.apply(fs, arguments)
}
const [offset, length] = entry
const resourceOffset = resourceStart + offset
const encoding = fileOpts(options).encoding
const fd = originalFsMethods.openSync(process.execPath, 'r')
const result = Buffer.alloc(length)
originalFsMethods.readSync(fd, result, 0, length, resourceOffset)
originalFsMethods.closeSync(fd)
return encoding ? result.toString(encoding) : result
},
statSync: function statSync(filepath: string | Buffer) {
const stat = ownStat(filepath)
if (stat) {
return stat
}
return originalFsMethods.statSync.apply(fs, arguments)
},
stat: function stat(filepath: string | Buffer, callback: any) {
const stat = ownStat(filepath)
if (stat) {
process.nextTick(() => {
callback(null, stat)
})
} else {
return originalFsMethods.stat.apply(fs, arguments)
}
}
}
if (typeof fs.exists === 'function') {
nfs.exists = function(filepath: string, cb: Function) {
cb = cb || noop
const exists = nfs.existsSync(filepath)
process.nextTick(() => cb(exists))
}
}
const patches = (process as any).nexe.patches || {}
delete (process as any).nexe
patches.internalModuleReadFile = function(this: any, original: any, ...args: any[]) {
const [filepath] = args
setupManifest()
if (manifest[filepath]) {
log('read (hit) ' + filepath)
return nfs.readFileSync(filepath, 'utf-8')
}
log('read (miss) ' + filepath)
return original.call(this, ...args)
}
patches.internalModuleReadJSON = patches.internalModuleReadFile
patches.internalModuleStat = function(this: any, original: any, ...args: any[]) {
setupManifest()
const [filepath] = args
if (manifest[filepath]) {
log('stat (hit) ' + filepath + ' ' + 0)
return 0
}
if (directories[filepath]) {
log('stat dir (hit) ' + filepath + ' ' + 1)
return 1
}
const res = original.call(this, ...args)
if (res === 0) {
log('stat (miss) ' + filepath + ' ' + res)
} else if (res === 1) {
log('stat dir (miss) ' + filepath + ' ' + res)
} else {
log('stat (fail) ' + filepath + ' ' + res)
}
return res
}
if (typeof fs.exists === 'function') {
nfs.exists = function(filepath: string, cb: Function) {
cb = cb || noop
const exists = nfs.existsSync(filepath)
if (!exists) {
return originalFsMethods.exists(filepath, cb)
}
process.nextTick(() => cb(exists))
}
}
if (typeof fs.copyFile === 'function') {
nfs.copyFile = function(filepath: string, dest: string, flags: number, callback: Function) {
setupManifest()
const entry = manifest[getKey(filepath)]
if (!entry) {
return originalFsMethods.copyFile.apply(fs, arguments)
}
if (typeof flags === 'function') {
callback = flags
flags = 0
}
nfs.readFile(filepath, (err: any, buffer: any) => {
if (err) {
return callback(err)
}
originalFsMethods.writeFile(dest, buffer, (err: any) => {
if (err) {
return callback(err)
}
callback(null)
})
})
}
nfs.copyFileSync = function(filepath: string, dest: string) {
setupManifest()
const entry = manifest[getKey(filepath)]
if (!entry) {
return originalFsMethods.copyFileSync.apply(fs, arguments)
}
return originalFsMethods.writeFileSync(dest, nfs.readFileSync(filepath))
}
}
Object.assign(fs, nfs)
lazyRestoreFs = () => { lazyRestoreFs = () => {
Object.assign(fs, originalFsMethods) Object.keys(nfs).forEach(key => {
Object.assign(patches, originalPatches) fs[key] = originalFsMethods[key]
})
lazyRestoreFs = () => {} lazyRestoreFs = () => {}
} }
return true
} }
function restoreFs() { function restoreFs() {
+3 -3
View File
@@ -24,7 +24,7 @@ export class Logger {
this.ora = ora({ this.ora = ora({
text: 'Starting...', text: 'Starting...',
color: 'blue', color: 'blue',
spinner: 'dots', spinner: 'dots'
}) })
this.ora.stop() this.ora.stop()
} }
@@ -35,7 +35,7 @@ export class Logger {
flush() { flush() {
!this.silent && this.ora.succeed() !this.silent && this.ora.succeed()
return new Promise((resolve) => setTimeout(resolve, frameLength)) return new Promise(resolve => setTimeout(resolve, frameLength))
} }
_write(update: string, color = 'green') { _write(update: string, color = 'green') {
@@ -66,7 +66,7 @@ export class Logger {
modify: this.modify, modify: this.modify,
log: this.verbose ? this.write : this.modify, log: this.verbose ? this.write : this.modify,
pause: () => this.ora.stopAndPersist(), pause: () => this.ora.stopAndPersist(),
resume: () => this.ora.start(), resume: () => this.ora.start()
} as LogStep } as LogStep
} }
} }
+2 -2
View File
@@ -15,7 +15,7 @@ async function compile(
compilerOptions?: Partial<NexeOptions>, compilerOptions?: Partial<NexeOptions>,
callback?: (err: Error | null) => void callback?: (err: Error | null) => void
) { ) {
let error: Error | null = null, let error = null,
options: NexeOptions | null = null, options: NexeOptions | null = null,
compiler: NexeCompiler | null = null compiler: NexeCompiler | null = null
@@ -32,7 +32,7 @@ async function compile(
options.build ? [artifacts, ...patches, ...(options.patches as NexePatch[])] : [], options.build ? [artifacts, ...patches, ...(options.patches as NexePatch[])] : [],
options.plugins as NexePatch[] options.plugins as NexePatch[]
)(compiler) )(compiler)
} catch (e: any) { } catch (e) {
error = e error = e
} }
+22 -27
View File
@@ -1,4 +1,4 @@
import parseArgv from 'minimist' import * as parseArgv from 'minimist'
import { NexeCompiler, NexeError } from './compiler' import { NexeCompiler, NexeError } from './compiler'
import { isWindows, STDIN_FLAG } from './util' import { isWindows, STDIN_FLAG } from './util'
import { basename, extname, join, isAbsolute, resolve } from 'path' import { basename, extname, join, isAbsolute, resolve } from 'path'
@@ -9,7 +9,7 @@ import { resolveSync } from 'resolve-dependencies'
const caw = require('caw') const caw = require('caw')
const c = process.platform === 'win32' ? chalk.constructor({ enabled: false }) : chalk const c = process.platform === 'win32' ? chalk.constructor({ enabled: false }) : chalk
export const version = '{{ version }}' export const version = '{{replace:0}}'
export interface NexePatch { export interface NexePatch {
(compiler: NexeCompiler, next: () => Promise<void>): Promise<void> (compiler: NexeCompiler, next: () => Promise<void>): Promise<void>
@@ -21,7 +21,6 @@ export interface NexeOptions {
output: string output: string
targets: (string | NexeTarget)[] targets: (string | NexeTarget)[]
name: string name: string
remote: string
asset: string asset: string
cwd: string cwd: string
fs: boolean | string[] fs: boolean | string[]
@@ -71,18 +70,17 @@ const defaults = {
build: false, build: false,
bundle: true, bundle: true,
patches: [], patches: [],
plugins: [], plugins: []
remote: 'https://github.com/nexe/nexe/releases/download/v3.3.3/',
} }
const alias = { const alias = {
i: 'input', i: 'input',
o: 'output', o: 'output',
v: 'version', v: 'version',
a: 'asset',
t: 'target', t: 'target',
b: 'build', b: 'build',
n: 'name', n: 'name',
r: 'resource', r: 'resource',
a: 'asset',
p: 'python', p: 'python',
f: 'flag', f: 'flag',
c: 'configure', c: 'configure',
@@ -90,7 +88,7 @@ const alias = {
h: 'help', h: 'help',
l: 'loglevel', l: 'loglevel',
'fake-argv': 'fakeArgv', 'fake-argv': 'fakeArgv',
'gh-token': 'ghToken', 'gh-token': 'ghToken'
} }
const argv = parseArgv(process.argv, { alias, default: { ...defaults } }) const argv = parseArgv(process.argv, { alias, default: { ...defaults } })
let help = ` let help = `
@@ -103,18 +101,18 @@ ${c.bold('nexe <entry-file> [options]')}
-t --target -- node version description -t --target -- node version description
-n --name -- main app module name -n --name -- main app module name
-r --resource -- *embed files (glob) within the binary -r --resource -- *embed files (glob) within the binary
--remote -- alternate location (URL) to download pre-built base (nexe) binaries from -a --asset -- alternate asset path, file or url pointing to a base (nexe) binary
--plugin -- extend nexe runtime behavior --plugin -- extend nexe runtime behavior
${c.underline.bold('Building from source:')} ${c.underline.bold('Building from source:')}
-b --build -- build from source -b --build -- build from source
-p --python -- python3 (as python) executable path -p --python -- python2 (as python) executable path
-f --flag -- *v8 flags to include during compilation -f --flag -- *v8 flags to include during compilation
-c --configure -- *arguments to the configure step -c --configure -- *arguments to the configure step
-m --make -- *arguments to the make/build step -m --make -- *arguments to the make/build step
--patch -- module with middleware default export for adding a build patch --patch -- module with middleware default export for adding a build patch
--no-mangle -- used when generating base binaries, or when patching _third_party_main manually --no-mangle -- used when generating base binaries, or when patching _third_party_main manually.
--snapshot -- path to a warmup snapshot --snapshot -- path to a warmup snapshot
--ico -- file name for alternate icon file (windows) --ico -- file name for alternate icon file (windows)
--rc-* -- populate rc file options (windows) --rc-* -- populate rc file options (windows)
@@ -135,7 +133,7 @@ ${c.bold('nexe <entry-file> [options]')}
help = EOL + help + EOL help = EOL + help + EOL
function flatten(...args: any[]): string[] { function flatten(...args: any[]): string[] {
return ([] as string[]).concat(...args).filter((x) => x) return ([] as string[]).concat(...args).filter(x => x)
} }
/** /**
@@ -146,13 +144,16 @@ function flatten(...args: any[]): string[] {
*/ */
function extractCliMap(match: RegExp, options: any) { function extractCliMap(match: RegExp, options: any) {
return Object.keys(options) return Object.keys(options)
.filter((x) => match.test(x)) .filter(x => match.test(x))
.reduce((map: any, option: any) => { .reduce(
const key = option.split('-')[1] (map: any, option: any) => {
map[key] = options[option] const key = option.split('-')[1]
delete options[option] map[key] = options[option]
return map delete options[option]
}, {} as any) return map
},
{} as any
)
} }
function extractLogLevel(options: NexeOptions) { function extractLogLevel(options: NexeOptions) {
@@ -251,19 +252,13 @@ function normalizeOptions(input?: Partial<NexeOptions>): NexeOptions {
options.configure = flatten(options.configure) options.configure = flatten(options.configure)
options.resources = flatten(opts.resource, options.resources) options.resources = flatten(opts.resource, options.resources)
if (!options.remote.endsWith('/')) {
options.remote += '/'
}
options.downloadOptions = options.downloadOptions || {} options.downloadOptions = options.downloadOptions || {}
options.downloadOptions.headers = options.downloadOptions.headers || {} options.downloadOptions.headers = options.downloadOptions.headers || {}
options.downloadOptions.headers['User-Agent'] = 'nexe (https://www.npmjs.com/package/nexe)' options.downloadOptions.headers['User-Agent'] = 'nexe (https://www.npmjs.com/package/nexe)'
options.downloadOptions.agent = process.env.HTTPS_PROXY options.downloadOptions.agent = process.env.HTTPS_PROXY
? caw(process.env.HTTPS_PROXY, { protocol: 'https' }) ? caw(process.env.HTTPS_PROXY, { protocol: 'https' })
: options.downloadOptions.agent || require('https').globalAgent : options.downloadOptions.agent || require('https').globalAgent
options.downloadOptions.rejectUnauthorized = process.env.NODE_TLS_REJECT_UNAUTHORIZED options.downloadOptions.rejectUnauthorized = process.env.HTTPS_PROXY ? false : true
? false
: true
options.rc = options.rc || extractCliMap(/^rc-.*/, options) options.rc = options.rc || extractCliMap(/^rc-.*/, options)
options.output = options.output =
@@ -297,8 +292,8 @@ function normalizeOptions(input?: Partial<NexeOptions>): NexeOptions {
} }
Object.keys(alias) Object.keys(alias)
.filter((k) => k !== 'rc') .filter(k => k !== 'rc')
.forEach((x) => delete opts[x]) .forEach(x => delete opts[x])
return options return options
} }
+6 -6
View File
@@ -20,13 +20,13 @@ const footer = tailWindow.slice(footerPosition, footerPosition + 32),
Object.defineProperty( Object.defineProperty(
process, process,
'__nexe', '__nexe',
(function () { (function() {
let nexeHeader: any = null let nexeHeader: any = null
return { return {
get: function () { get: function() {
return nexeHeader return nexeHeader
}, },
set: function (value: any) { set: function(value: any) {
if (nexeHeader) { if (nexeHeader) {
throw new Error('This property is readonly') throw new Error('This property is readonly')
} }
@@ -37,13 +37,13 @@ Object.defineProperty(
contentSize, contentSize,
contentStart, contentStart,
resourceSize, resourceSize,
resourceStart, resourceStart
}, }
}) })
Object.freeze(nexeHeader) Object.freeze(nexeHeader)
}, },
enumerable: false, enumerable: false,
configurable: false, configurable: false
} }
})() })()
) )
+4 -4
View File
@@ -9,11 +9,11 @@ export default async function disableNodeCli(compiler: NexeCompiler, next: () =>
if (semverGt(compiler.target.version, '11.6.0')) { if (semverGt(compiler.target.version, '11.6.0')) {
await compiler.replaceInFileAsync( await compiler.replaceInFileAsync(
'src/node.cc', 'src/node.cc',
/(?<!int )ProcessGlobalArgs\(argv[^;]*;/gm, /(?<!int )ProcessGlobalArgs\(/g,
'0;/*$&*/' '0;//ProcessGlobalArgs('
) )
} else if (semverGt(compiler.target.version, '10.9')) { } else if (semverGt(compiler.target.version, '10.9')) {
await compiler.replaceInFileAsync('src/node.cc', /(?<!void )ProcessArgv\(argv/g, '//$&') await compiler.replaceInFileAsync('src/node.cc', /(?<!void )ProcessArgv\(/g, '//ProcessArgv(')
} else if (semverGt(compiler.target.version, '9.999')) { } else if (semverGt(compiler.target.version, '9.999')) {
await compiler.replaceInFileAsync( await compiler.replaceInFileAsync(
'src/node.cc', 'src/node.cc',
@@ -21,7 +21,7 @@ export default async function disableNodeCli(compiler: NexeCompiler, next: () =>
'int i = v8_argc; i < v8_argc; i++' 'int i = v8_argc; i < v8_argc; i++'
) )
let matches = 0 let matches = 0
await compiler.replaceInFileAsync('src/node.cc', /v8_argc > 1/g, (match) => { await compiler.replaceInFileAsync('src/node.cc', /v8_argc > 1/g, match => {
if (matches++) { if (matches++) {
return 'false' return 'false'
} }
+2 -2
View File
@@ -13,8 +13,8 @@ export default async function nodeGyp(
` `
${nodeGypMarker} ${nodeGypMarker}
${files ${files
.filter((x) => x.filename.startsWith('lib')) .filter(x => x.filename.startsWith('lib'))
.map((x) => `'${x.filename}'`) .map(x => `'${x.filename}'`)
.toString()}, .toString()},
`.trim() `.trim()
) )
+7 -8
View File
@@ -8,19 +8,18 @@ export default async function nodeRc(compiler: NexeCompiler, next: () => Promise
const file = await compiler.readFileAsync('src/res/node.rc') const file = await compiler.readFileAsync('src/res/node.rc')
Object.keys(options).forEach((key) => { Object.keys(options).forEach(key => {
let value = options[key] let value = options[key]
const isVar = /^[A-Z_]+$/.test(value) const isVar = /^[A-Z_]+$/.test(value)
value = isVar ? value : `"${value}"` value = isVar ? value : `"${value}"`
file.contents = file.contents file.contents = file.contents.replace(
.toString() new RegExp(`VALUE "${key}",.*`),
.replace(new RegExp(`VALUE "${key}",.*`), `VALUE "${key}", ${value}`) `VALUE "${key}", ${value}`
)
}) })
;['PRODUCTVERSION', 'FILEVERSION'].forEach((x) => { ;['PRODUCTVERSION', 'FILEVERSION'].forEach(x => {
if (options[x]) { if (options[x]) {
file.contents = file.contents file.contents = file.contents.replace(new RegExp(x + ' .*$', 'm'), `${x} ${options[x]}`)
.toString()
.replace(new RegExp(x + ' .*$', 'm'), `${x} ${options[x]}`)
} }
}) })
+1 -1
View File
@@ -2,7 +2,7 @@ import { resolve } from 'path'
import { NexeCompiler } from '../compiler' import { NexeCompiler } from '../compiler'
import { semverGt } from '../util' import { semverGt } from '../util'
export default async function (compiler: NexeCompiler, next: () => Promise<void>) { export default async function(compiler: NexeCompiler, next: () => Promise<void>) {
const { snapshot, warmup, cwd } = compiler.options const { snapshot, warmup, cwd } = compiler.options
if (!snapshot) { if (!snapshot) {
+23 -55
View File
@@ -1,5 +1,5 @@
import { NexeCompiler } from '../compiler' import { NexeCompiler } from '../compiler'
import { parse } from 'meriyah' import { parse } from 'cherow'
import { wrap, semverGt } from '../util' import { wrap, semverGt } from '../util'
function walkSome(node: any, visit: Function) { function walkSome(node: any, visit: Function) {
@@ -29,8 +29,6 @@ export default async function main(compiler: NexeCompiler, next: () => Promise<v
if (version.startsWith('4.')) { if (version.startsWith('4.')) {
bootFile = 'src/node.js' bootFile = 'src/node.js'
} else if (semverGt(version, '18.11.99')) {
bootFile = 'lib/internal/process/pre_execution.js'
} else if (semverGt(version, '11.99')) { } else if (semverGt(version, '11.99')) {
bootFile = 'lib/internal/bootstrap/pre_execution.js' bootFile = 'lib/internal/bootstrap/pre_execution.js'
} else if (semverGt(version, '9.10.1')) { } else if (semverGt(version, '9.10.1')) {
@@ -38,11 +36,13 @@ export default async function main(compiler: NexeCompiler, next: () => Promise<v
} }
const file = await compiler.readFileAsync(bootFile), const file = await compiler.readFileAsync(bootFile),
ast = parse(file.contents.toString(), { ast = parse(file.contents, {
loc: true,
tolerant: true,
next: true, next: true,
globalReturn: true, globalReturn: true,
loc: true, node: true,
specDeviation: true, skipShebang: true
}), }),
location = { start: { line: 0 } } location = { start: { line: 0 } }
@@ -54,64 +54,32 @@ export default async function main(compiler: NexeCompiler, next: () => Promise<v
} }
}) })
const fileLines = file.contents.toString().split('\n') const fileLines = file.contents.split('\n')
if (semverGt(version, '19.99')) { fileLines.splice(
await compiler.replaceInFileAsync( location.start.line,
'lib/internal/modules/cjs/loader.js', 0,
"'use strict';", '{{replace:lib/fs/bootstrap.js}}' +
"'use strict';\n" + '{{ file("lib/fs/bootstrap.js") }}' + '\n' '\n' +
) (semverGt(version, '11.99') ? 'expandArgv1 = false;\n' : '')
fileLines.splice(location.start.line, 0, 'expandArgv1 = false;') )
} else {
fileLines.splice(
location.start.line,
0,
'{{ file("lib/fs/bootstrap.js") }}' +
'\n' +
(semverGt(version, '11.99') ? 'expandArgv1 = false;\n' : '')
)
}
file.contents = fileLines.join('\n') file.contents = fileLines.join('\n')
if (semverGt(version, '11.99')) { if (semverGt(version, '11.99')) {
if (semverGt(version, '12.17.99')) {
await compiler.replaceInFileAsync(
bootFile,
'initializeFrozenIntrinsics();',
'initializeFrozenIntrinsics();\n' + wrap('{{ file("lib/patches/boot-nexe.js") }}')
)
} else {
await compiler.replaceInFileAsync(
bootFile,
'initializePolicy();',
'initializePolicy();\n' + wrap('{{ file("lib/patches/boot-nexe.js") }}')
)
}
await compiler.replaceInFileAsync( await compiler.replaceInFileAsync(
bootFile, bootFile,
'assert(!CJSLoader.hasLoadedAnyUserCJSModule)', 'initializePolicy();',
'/*assert(!CJSLoader.hasLoadedAnyUserCJSModule)*/' 'initializePolicy();\n' + wrap('{{replace:lib/patches/boot-nexe.js}}')
)
await compiler.replaceInFileAsync(
'src/node.cc',
'MaybeLocal<Value> StartMainThreadExecution(Environment* env) {',
'MaybeLocal<Value> StartMainThreadExecution(Environment* env) {\n' +
' return StartExecution(env, "internal/main/run_main_module");\n'
) )
const { contents: nodeccContents } = await compiler.readFileAsync('src/node.cc')
if (nodeccContents.includes('if (env->worker_context() != nullptr) {')) {
await compiler.replaceInFileAsync(
'src/node.cc',
'if (env->worker_context() != nullptr) {',
'if (env->worker_context() == nullptr) {\n' +
' return StartExecution(env, "internal/main/run_main_module"); } else {\n'
)
} else {
await compiler.replaceInFileAsync(
'src/node.cc',
'MaybeLocal<Value> StartMainThreadExecution(Environment* env) {',
'MaybeLocal<Value> StartMainThreadExecution(Environment* env) {\n' +
' return StartExecution(env, "internal/main/run_main_module");\n'
)
}
} else { } else {
await compiler.setFileContentsAsync( await compiler.setFileContentsAsync(
'lib/_third_party_main.js', 'lib/_third_party_main.js',
'{{ file("lib/patches/boot-nexe.js") }}' '{{replace:lib/patches/boot-nexe.js}}'
) )
} }
return next() return next()
+9 -11
View File
@@ -2,7 +2,7 @@ import got = require('got')
import { platforms, architectures, NexeTarget, getTarget, targetsEqual } from './target' import { platforms, architectures, NexeTarget, getTarget, targetsEqual } from './target'
export { NexeTarget } export { NexeTarget }
const versionsToSkip = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 13, 15, 17, 19, 21, 22] const versionsToSkip = [0, 1, 2, 3, 4, 5, 7, 9, 11]
export interface GitAsset { export interface GitAsset {
name: string name: string
url: string url: string
@@ -25,21 +25,19 @@ async function getJson<T>(url: string, options?: any) {
} }
function isBuildableVersion(version: string) { function isBuildableVersion(version: string) {
if (version === '12.11.0') { const major = Number(version.split('.')[0])
return false return !versionsToSkip.includes(major) || version === '4.8.4'
}
return !versionsToSkip.includes(Number(version.split('.')[0]))
} }
export function getLatestGitRelease(options?: any) { export function getV3GitRelease(options?: any) {
return getJson<GitRelease>('https://api.github.com/repos/nexe/nexe/releases/latest', options) return getJson<GitRelease>('https://api.github.com/repos/nexe/nexe/releases/7567580/', options)
} }
export async function getUnBuiltReleases(options?: any) { export async function getUnBuiltReleases(options?: any) {
const nodeReleases = await getJson<NodeRelease[]>( const nodeReleases = await getJson<NodeRelease[]>(
'https://nodejs.org/download/release/index.json' 'https://nodejs.org/download/release/index.json'
) )
const existingVersions = (await getLatestGitRelease(options)).assets.map((x) => getTarget(x.name)) const existingVersions = (await getV3GitRelease(options)).assets.map(x => getTarget(x.name))
const versionMap: { [key: string]: true } = {} const versionMap: { [key: string]: true } = {}
return nodeReleases return nodeReleases
@@ -49,8 +47,8 @@ export async function getUnBuiltReleases(options?: any) {
return versions return versions
} }
versionMap[version] = true versionMap[version] = true
platforms.forEach((platform) => { platforms.forEach(platform => {
architectures.forEach((arch) => { architectures.forEach(arch => {
if (arch === 'x86' && platform === 'mac') return if (arch === 'x86' && platform === 'mac') return
if (arch.includes('arm')) return if (arch.includes('arm')) return
versions.push(getTarget({ platform, arch, version })) versions.push(getTarget({ platform, arch, version }))
@@ -58,5 +56,5 @@ export async function getUnBuiltReleases(options?: any) {
}) })
return versions return versions
}, []) }, [])
.filter((x) => !existingVersions.some((t) => targetsEqual(t, x))) .filter(x => !existingVersions.some(t => targetsEqual(t, x)))
} }
+13 -12
View File
@@ -1,28 +1,29 @@
import { join, dirname } from 'path' import { join, dirname } from 'path'
import * as fs from 'fs' import { readdir, unlink } from 'fs'
import { promisify } from 'util'
import { readFileAsync, writeFileAsync, isDirectoryAsync } from '../util' import { readFileAsync, writeFileAsync, isDirectoryAsync } from '../util'
import mkdirpAsync = require('mkdirp') import * as mkdirp from 'mkdirp'
import { NexeCompiler } from '../compiler' import { NexeCompiler } from '../compiler'
import pify = require('pify')
const unlinkAsync = promisify(fs.unlink), const mkdirpAsync = pify(mkdirp)
readdirAsync = promisify(fs.readdir) const unlinkAsync = pify(unlink)
const readdirAsync = pify(readdir)
function readDirAsync(dir: string): Promise<string[]> { function readDirAsync(dir: string): Promise<string[]> {
return readdirAsync(dir).then((paths) => { return readdirAsync(dir).then(paths => {
return Promise.all( return Promise.all(
paths.map((file: string) => { paths.map((file: string) => {
const path = join(dir, file) const path = join(dir, file)
return isDirectoryAsync(path).then((x) => (x ? readDirAsync(path) : (path as any))) return isDirectoryAsync(path).then(x => (x ? readDirAsync(path) : (path as any)))
}) })
).then((result) => { ).then(result => {
return [].concat(...(result as any)) return [].concat(...(result as any))
}) })
}) })
} }
function maybeReadFileContentsAsync(file: string) { function maybeReadFileContentsAsync(file: string) {
return readFileAsync(file, 'utf-8').catch((e) => { return readFileAsync(file, 'utf-8').catch(e => {
if (e.code === 'ENOENT') { if (e.code === 'ENOENT') {
return '' return ''
} }
@@ -46,16 +47,16 @@ export default async function artifacts(compiler: NexeCompiler, next: () => Prom
const tmpFiles = await readDirAsync(temp) const tmpFiles = await readDirAsync(temp)
await Promise.all( await Promise.all(
tmpFiles.map(async (path) => { tmpFiles.map(async path => {
return compiler.writeFileAsync(path.replace(temp, ''), await readFileAsync(path, 'utf-8')) return compiler.writeFileAsync(path.replace(temp, ''), await readFileAsync(path, 'utf-8'))
}) })
) )
await next() await next()
await Promise.all(tmpFiles.map((x) => unlinkAsync(x))) await Promise.all(tmpFiles.map(x => unlinkAsync(x)))
return Promise.all( return Promise.all(
compiler.files.map(async (file) => { compiler.files.map(async file => {
const sourceFile = join(src, file.filename) const sourceFile = join(src, file.filename)
const tempFile = join(temp, file.filename) const tempFile = join(temp, file.filename)
const fileContents = await maybeReadFileContentsAsync(sourceFile) const fileContents = await maybeReadFileContentsAsync(sourceFile)
+13 -20
View File
@@ -1,12 +1,13 @@
import { NexeCompiler, NexeError } from '../compiler' import { NexeCompiler, NexeError } from '../compiler'
import { resolve, relative } from 'path' import { resolve, relative } from 'path'
import { each } from '@calebboyd/semaphore'
import resolveFiles, { resolveSync } from 'resolve-dependencies' import resolveFiles, { resolveSync } from 'resolve-dependencies'
import { dequote, STDIN_FLAG, semverGt, each } from '../util' import { dequote, STDIN_FLAG, semverGt } from '../util'
import { Readable } from 'stream' import { Readable } from 'stream'
function getStdIn(stdin: Readable): Promise<string> { function getStdIn(stdin: Readable): Promise<string> {
let out = '' let out = ''
return new Promise((resolve) => { return new Promise(resolve => {
stdin stdin
.setEncoding('utf8') .setEncoding('utf8')
.on('readable', () => { .on('readable', () => {
@@ -25,7 +26,7 @@ function getStdIn(stdin: Readable): Promise<string> {
} }
export default async function bundle(compiler: NexeCompiler, next: any) { export default async function bundle(compiler: NexeCompiler, next: any) {
const { bundle: doBundle, cwd, input: inputPath } = compiler.options const { bundle, cwd, input: inputPath } = compiler.options
let input = inputPath let input = inputPath
compiler.entrypoint = './' + relative(cwd, input) compiler.entrypoint = './' + relative(cwd, input)
@@ -35,14 +36,14 @@ export default async function bundle(compiler: NexeCompiler, next: any) {
compiler.startup = ';require("module").runMain();' compiler.startup = ';require("module").runMain();'
} }
if (!doBundle) { if (!bundle) {
await compiler.addResource(resolve(cwd, input)) await compiler.addResource(resolve(cwd, input))
return next() return next()
} }
let code = '' let code = ''
if (typeof doBundle === 'string') { if (typeof bundle === 'string') {
code = await require(doBundle).createBundle(compiler.options) code = await require(bundle).createBundle(compiler.options)
} }
if (input === STDIN_FLAG && (code = code || dequote(await getStdIn(process.stdin)))) { if (input === STDIN_FLAG && (code = code || dequote(await getStdIn(process.stdin)))) {
@@ -61,28 +62,20 @@ export default async function bundle(compiler: NexeCompiler, next: any) {
compiler.entrypoint = './' + relative(cwd, input) compiler.entrypoint = './' + relative(cwd, input)
} }
const step = compiler.log.step('Resolving dependencies...')
const { files, warnings } = await resolveFiles( const { files, warnings } = await resolveFiles(
input, input,
...Object.keys(compiler.bundle.list).filter((x) => x.endsWith('.js') || x.endsWith('.mjs')), ...Object.keys(compiler.bundle.index).filter(x => x.endsWith('.js')),
{ cwd, expand: 'variable', loadContent: false } { cwd, expand: 'all', loadContent: false }
) )
if ( if (
warnings.filter((x) => x.startsWith('Error parsing file') && !x.includes('node_modules')).length warnings.filter(x => x.startsWith('Error parsing file') && !x.includes('node_modules')).length
) { ) {
throw new NexeError('Parsing Error:\n' + warnings.join('\n')) throw new NexeError('Parsing Error:\n' + warnings.join('\n'))
} }
//TODO: warnings.forEach((x) => console.log(x)) await each(Object.keys(files), (filename: string) => compiler.addResource(filename), {
concurrency: 10
await Promise.all( })
Object.entries(files).map(([key]) => {
step.log(`Including dependency: ${key}`)
return compiler.addResource(key)
})
)
return next() return next()
} }
+4 -4
View File
@@ -11,7 +11,7 @@ function fetchNodeSourceAsync(dest: string, url: string, step: LogStep, options
.on('response', (res: IncomingMessage) => { .on('response', (res: IncomingMessage) => {
const total = +res.headers['content-length']! const total = +res.headers['content-length']!
let current = 0 let current = 0
res.on('data', (data) => { res.on('data', data => {
current += data.length current += data.length
setText((current / total) * 100) setText((current / total) * 100)
if (current === total) { if (current === total) {
@@ -32,13 +32,13 @@ async function fetchPrebuiltBinary(compiler: NexeCompiler, step: any) {
(res: IncomingMessage) => { (res: IncomingMessage) => {
const total = +res.headers['content-length']! const total = +res.headers['content-length']!
let current = 0 let current = 0
res.on('data', (data) => { res.on('data', data => {
current += data.length current += data.length
step!.modify(`Downloading...${((current / total) * 100).toFixed()}%`) step!.modify(`Downloading...${((current / total) * 100).toFixed()}%`)
}) })
} }
) )
} catch (e: any) { } catch (e) {
if (e.statusCode === 404) { if (e.statusCode === 404) {
throw new NexeError(`${remoteAsset} is not available, create it using the --build flag`) throw new NexeError(`${remoteAsset} is not available, create it using the --build flag`)
} else { } else {
@@ -58,7 +58,7 @@ export default async function downloadNode(compiler: NexeCompiler, next: () => P
{ sourceUrl, downloadOptions, build } = compiler.options, { sourceUrl, downloadOptions, build } = compiler.options,
url = sourceUrl || `https://nodejs.org/dist/v${version}/node-v${version}.tar.gz`, url = sourceUrl || `https://nodejs.org/dist/v${version}/node-v${version}.tar.gz`,
step = log.step( step = log.step(
`Downloading ${build ? '' : 'pre-built '}Node.js${build ? ` source from: ${url}` : ''}` `Downloading ${build ? '' : 'pre-built '}Node.js${build ? `source from: ${url}` : ''}`
), ),
exeLocation = compiler.getNodeExecutableLocation(build ? undefined : target), exeLocation = compiler.getNodeExecutableLocation(build ? undefined : target),
downloadExists = await pathExistsAsync(build ? src : exeLocation) downloadExists = await pathExistsAsync(build ? src : exeLocation)
+9 -12
View File
@@ -1,25 +1,22 @@
import { each } from '../util' import { isDirectoryAsync, each } from '../util'
import globs from 'globby' import globs from 'globby'
import { resolve } from 'path'
import { NexeCompiler } from '../compiler' import { NexeCompiler } from '../compiler'
export default async function resource(compiler: NexeCompiler, next: () => Promise<any>) { export default async function resource(compiler: NexeCompiler, next: () => Promise<any>) {
const { cwd, resources } = compiler.options const { cwd } = compiler.options
if (!resources.length) { if (!compiler.options.resources.length) {
return next() return next()
} }
const step = compiler.log.step('Bundling Resources...') const step = compiler.log.step('Bundling Resources...')
let count = 0 let count = 0
await each(globs(compiler.options.resources, { cwd }), async file => {
// workaround for https://github.com/sindresorhus/globby/issues/127 if (await isDirectoryAsync(file)) {
// and https://github.com/mrmlnc/fast-glob#pattern-syntax return
const resourcesWithForwardSlashes = resources.map((r) => r.replace(/\\/g, '/')) }
await each(globs(resourcesWithForwardSlashes, { cwd, onlyFiles: true }), async (file) => {
count++ count++
step.log(`Including file: ${file}`) step.log(`Including file: ${file}`)
await compiler.addResource(resolve(cwd, file)) await compiler.addResource(file)
}) })
step.log(`Included ${count} file(s)`) step.log(`Included ${count} file(s). ${(compiler.resourceSize / 1e6).toFixed(3)} MB`)
return next() return next()
} }
+7 -13
View File
@@ -1,21 +1,13 @@
import { NexeCompiler } from '../compiler' import { NexeCompiler } from '../compiler'
import { wrap } from '../util' import { wrap } from '../util'
export default async function (compiler: NexeCompiler, next: () => Promise<void>) { export default function(compiler: NexeCompiler, next: () => Promise<void>) {
await next()
compiler.shims.push( compiler.shims.push(
wrap( wrap(
[ `process.__nexe = ${JSON.stringify(compiler.binaryConfiguration)};\n` +
'process.__nexe = {};', '{{replace:lib/fs/patch.js}}' +
'const fsPatcher = (function() {', '\nshimFs(process.__nexe)' +
'const module = {exports: {}};', `\n${compiler.options.fs ? '' : 'restoreFs()'}`
'const exports = module.exports;',
'{{file("lib/fs/patch.bundle.js")}}',
'return module.exports;',
'})()',
'fsPatcher.shimFs(process.__nexe);',
compiler.options.fs ? '' : 'restoreFs();',
].join('\n')
//TODO support only restoring specific methods //TODO support only restoring specific methods
) )
) )
@@ -40,4 +32,6 @@ export default async function (compiler: NexeCompiler, next: () => Promise<void>
} }
`) `)
) )
return next()
} }
+10 -10
View File
@@ -1,8 +1,8 @@
export type NodePlatform = 'windows' | 'mac' | 'alpine' | 'linux' export type NodePlatform = 'windows' | 'mac' | 'alpine' | 'linux'
export type NodeArch = 'x86' | 'x64' | 'arm' | 'arm64' export type NodeArch = 'x86' | 'x64' | 'arm7l' | 'arm6l' | 'arm64'
const platforms: NodePlatform[] = ['windows', 'mac', 'alpine', 'linux'], const platforms: NodePlatform[] = ['windows', 'mac', 'alpine', 'linux'],
architectures: NodeArch[] = ['x86', 'x64', 'arm', 'arm64'] architectures: NodeArch[] = ['x86', 'x64', 'arm7l', 'arm6l', 'arm64']
export { platforms, architectures } export { platforms, architectures }
@@ -21,21 +21,21 @@ const prettyPlatform: { [key: string]: NodePlatform } = {
mac: 'mac', mac: 'mac',
linux: 'linux', linux: 'linux',
static: 'alpine', static: 'alpine',
alpine: 'alpine', alpine: 'alpine'
} }
const prettyArch: { [key: string]: NodeArch } = { const prettyArch: { [key: string]: NodeArch } = {
x86: 'x86', x86: 'x86',
arm6: 'arm', arm6: 'arm6l',
arm64: 'arm64', arm64: 'arm64',
arm6l: 'arm', arm6l: 'arm6l',
arm: 'arm', arm: 'arm7l',
arm7: 'arm', arm7: 'arm7l',
arm7l: 'arm', arm7l: 'arm7l',
amd64: 'x64', amd64: 'x64',
ia32: 'x86', ia32: 'x86',
x32: 'x86', x32: 'x86',
x64: 'x64', x64: 'x64'
} }
function isVersion(x: string) { function isVersion(x: string) {
@@ -80,7 +80,7 @@ export function getTarget(target: string | Partial<NexeTarget> = ''): NexeTarget
target target
.toLowerCase() .toLowerCase()
.split('-') .split('-')
.forEach((x) => { .forEach(x => {
if (isVersion(x)) { if (isVersion(x)) {
version = x.replace(/v/g, '') version = x.replace(/v/g, '')
} }
+12 -12
View File
@@ -1,9 +1,9 @@
import { readFile, writeFile, stat } from 'fs' import { readFile, writeFile, stat } from 'fs'
import { execFile } from 'child_process' import { execFile } from 'child_process'
import { promisify } from 'util' import pify = require('pify')
import rimraf = require('rimraf') import rimraf = require('rimraf')
const rimrafAsync = promisify(rimraf) const rimrafAsync = pify(rimraf)
export const STDIN_FLAG = '[stdin]' export const STDIN_FLAG = '[stdin]'
export async function each<T>( export async function each<T>(
@@ -26,7 +26,7 @@ function falseOnEnoent(e: any) {
} }
function padRight(str: string, l: number) { function padRight(str: string, l: number) {
return (str + ' '.repeat(l)).slice(0, l) return (str + ' '.repeat(l)).substr(0, l)
} }
const bound: MethodDecorator = function bound<T>( const bound: MethodDecorator = function bound<T>(
@@ -42,10 +42,10 @@ const bound: MethodDecorator = function bound<T>(
Object.defineProperty(this, propertyKey as string, { Object.defineProperty(this, propertyKey as string, {
configurable, configurable,
value, value,
writable: true, writable: true
}) })
return value return value
}, }
} }
} }
@@ -64,21 +64,21 @@ export interface ReadFileAsync {
(path: string, encoding: string): Promise<string> (path: string, encoding: string): Promise<string>
} }
const readFileAsync = promisify(readFile) const readFileAsync = pify(readFile)
const writeFileAsync = promisify(writeFile) const writeFileAsync = pify(writeFile)
const statAsync = promisify(stat) const statAsync = pify(stat)
const execFileAsync = promisify(execFile) const execFileAsync = pify(execFile)
const isWindows = process.platform === 'win32' const isWindows = process.platform === 'win32'
function pathExistsAsync(path: string) { function pathExistsAsync(path: string) {
return statAsync(path) return statAsync(path)
.then((x) => true) .then(x => true)
.catch(falseOnEnoent) .catch(falseOnEnoent)
} }
function isDirectoryAsync(path: string) { function isDirectoryAsync(path: string) {
return statAsync(path) return statAsync(path)
.then((x) => x.isDirectory()) .then(x => x.isDirectory())
.catch(falseOnEnoent) .catch(falseOnEnoent)
} }
/** /**
@@ -109,5 +109,5 @@ export {
readFileAsync, readFileAsync,
pathExistsAsync, pathExistsAsync,
isDirectoryAsync, isDirectoryAsync,
writeFileAsync, writeFileAsync
} }
+16 -16
View File
@@ -1,9 +1,9 @@
import * as nexe from '../lib/nexe' import * as nexe from '../lib/nexe'
import { getUnBuiltReleases, getLatestGitRelease } from '../lib/releases' import { getUnBuiltReleases, getV3GitRelease } from '../lib/releases'
import { runDockerBuild } from './docker' import { runDockerBuild } from './docker'
import { getTarget, targetsEqual, NexeTarget } from '../lib/target' import { getTarget, targetsEqual, NexeTarget } from '../lib/target'
import { pathExistsAsync, readFileAsync, execFileAsync, semverGt } from '../lib/util' import { pathExistsAsync, readFileAsync, execFileAsync, semverGt } from '../lib/util'
import got from 'got' import got = require('got')
import { cpus } from 'os' import { cpus } from 'os'
const env = process.env, const env = process.env,
@@ -14,12 +14,12 @@ const env = process.env,
isMac = process.platform === 'darwin', isMac = process.platform === 'darwin',
headers = { headers = {
Authorization: 'token ' + env.GITHUB_TOKEN, Authorization: 'token ' + env.GITHUB_TOKEN,
'User-Agent': 'nexe (https://www.npmjs.com/package/nexe)', 'User-Agent': 'nexe (https://www.npmjs.com/package/nexe)'
} }
if (require.main === module) { if (require.main === module) {
if (!isPullRequest) { if (!isPullRequest) {
build().catch((x) => { build().catch(x => {
console.error(x) console.error(x)
process.exit(1) process.exit(1)
}) })
@@ -31,10 +31,10 @@ async function build() {
if (!releases.length) { if (!releases.length) {
return return
} }
const windowsBuild = releases.find((x) => x.platform === 'windows'), const windowsBuild = releases.find(x => x.platform === 'windows'),
macBuild = releases.find((x) => x.platform === 'mac'), macBuild = releases.find(x => x.platform === 'mac'),
linux = releases.find((x) => x.platform === 'linux'), linux = releases.find(x => x.platform === 'linux'),
alpine = releases.find((x) => x.platform === 'alpine') alpine = releases.find(x => x.platform === 'alpine')
let target: NexeTarget | undefined let target: NexeTarget | undefined
@@ -62,7 +62,7 @@ async function build() {
verbose: Boolean(env.NEXE_VERBOSE!), verbose: Boolean(env.NEXE_VERBOSE!),
target, target,
make: ['-j' + cpus().length], make: ['-j' + cpus().length],
output, output
} }
console.log('Building: ' + target + ' on ' + buildHost) console.log('Building: ' + target + ' on ' + buildHost)
const stop = keepalive() const stop = keepalive()
@@ -78,21 +78,21 @@ async function build() {
stop() stop()
if (await pathExistsAsync(output)) { if (await pathExistsAsync(output)) {
await assertNexeBinary(output) await assertNexeBinary(output)
const gitRelease = await getLatestGitRelease({ headers }), const gitRelease = await getV3GitRelease({ headers }),
unbuiltReleases = await getUnBuiltReleases({ headers }) unbuiltReleases = await getUnBuiltReleases({ headers })
if (!unbuiltReleases.some((x) => targetsEqual(x, target!))) { if (!unbuiltReleases.some(x => targetsEqual(x, target!))) {
console.log(`${target} already uploaded.`) console.log(`${target} already uploaded.`)
process.exit(0) process.exit(0)
return return
} }
await got(gitRelease.upload_url.split('{')[0], { await got(gitRelease.upload_url.split('{')[0], {
searchParams: { name: target.toString() }, query: { name: target.toString() },
body: await readFileAsync(output), body: await readFileAsync(output),
headers: { headers: {
...headers, ...headers,
'Content-Type': 'application/octet-stream', 'Content-Type': 'application/octet-stream'
}, }
}).catch((reason) => { }).catch(reason => {
console.log(reason && reason.response && reason.response.body) console.log(reason && reason.response && reason.response.body)
throw reason throw reason
}) })
@@ -107,7 +107,7 @@ function keepalive() {
} }
function assertNexeBinary(file: string) { function assertNexeBinary(file: string) {
return execFileAsync(file).catch((e) => { return execFileAsync(file).catch(e => {
if (e && e.stack && e.stack.includes('Invalid Nexe binary')) { if (e && e.stack && e.stack.includes('Invalid Nexe binary')) {
return return
} }
+18 -13
View File
@@ -1,12 +1,12 @@
import { NexeTarget, architectures } from '../lib/target' import { NexeTarget, architectures } from '../lib/target'
import { writeFileAsync, readFileAsync } from '../lib/util' import { writeFileAsync, readFileAsync } from '../lib/util'
import got from 'got' import got = require('got')
import execa = require('execa') import execa = require('execa')
import { appendFileSync } from 'fs' import { appendFileSync } from 'fs'
function alpine(target: NexeTarget) { function alpine(target: NexeTarget) {
return ` return `
FROM ${target.arch === 'x64' ? '' : 'i386/'}alpine:3.12 FROM ${target.arch === 'x64' ? '' : 'i386/'}alpine:3.4
RUN apk add --no-cache curl make gcc g++ binutils-gold python linux-headers paxctl libgcc libstdc++ git vim tar gzip wget RUN apk add --no-cache curl make gcc g++ binutils-gold python linux-headers paxctl libgcc libstdc++ git vim tar gzip wget
ENV NODE_VERSION=${target.version} ENV NODE_VERSION=${target.version}
ENV NEXE_VERSION=latest ENV NEXE_VERSION=latest
@@ -46,11 +46,11 @@ export async function runDockerBuild(target: NexeTarget) {
let output: any = [] let output: any = []
try { try {
output.push(await execa(`docker build -t nexe-docker .`, { shell: true })) output.push(await execa.shell(`docker build -t nexe-docker .`))
output.push(await execa(`docker run -d --name nexe nexe-docker sh`, { shell: true })) output.push(await execa.shell(`docker run -d --name nexe nexe-docker sh`))
output.push(await execa(`docker cp nexe:/out out`, { shell: true })) output.push(await execa.shell(`docker cp nexe:/out out`))
output.push(await execa(`docker rm nexe`, { shell: true })) output.push(await execa.shell(`docker rm nexe`))
} catch (e: any) { } catch (e) {
console.log('Error running docker') console.log('Error running docker')
appendFileSync(outFilename, e.message) appendFileSync(outFilename, e.message)
} finally { } finally {
@@ -58,11 +58,16 @@ export async function runDockerBuild(target: NexeTarget) {
appendFileSync(outFilename, x.stderr) appendFileSync(outFilename, x.stderr)
appendFileSync(outFilename, x.stdout) appendFileSync(outFilename, x.stdout)
}) })
await got(`https://transfer.sh/${Math.random().toString(36).substring(2)}.txt`, { await got(
body: await readFileAsync(outFilename), `https://transfer.sh/${Math.random()
method: 'PUT', .toString(36)
}) .substring(2)}.txt`,
.then((x) => console.log('Posted docker log: ', x.body)) {
.catch((e) => console.log('Error posting log', e)) body: await readFileAsync(outFilename),
method: 'PUT'
}
)
.then(x => console.log('Posted docker log: ', x.body))
.catch(e => console.log('Error posting log', e))
} }
} }
+15 -8
View File
@@ -1,10 +1,11 @@
import { writeFileSync, readFileSync } from 'fs' import { writeFileSync, readFileSync } from 'fs'
import { template } from 'lodash'
/** /**
* post build step to insert code files into code files. * post build step to insert code files into code files (naively).
* '{{replace:path/to/file}}' => "file contents"
* And the package.json version. * And the package.json version.
*/ */
//inject('plugins/nexe-daemon/lib/index.js')
cp('src/fs/package.json', 'lib/fs/package.json') cp('src/fs/package.json', 'lib/fs/package.json')
cp('src/fs/bootstrap.js', 'lib/fs/bootstrap.js') cp('src/fs/bootstrap.js', 'lib/fs/bootstrap.js')
cp('src/fs/README.md', 'lib/fs/README.md') cp('src/fs/README.md', 'lib/fs/README.md')
@@ -13,12 +14,18 @@ inject('lib/patches/third-party-main.js')
inject('lib/steps/shim.js') inject('lib/steps/shim.js')
inject('lib/options.js', JSON.stringify(require('../package.json').version)) inject('lib/options.js', JSON.stringify(require('../package.json').version))
function inject(filename: string, version?: string) { function inject(filename: string, ...replacements: string[]) {
const contents = template(readFileSync(filename, 'utf8'), { let contents = readFileSync(filename, 'utf8')
interpolate: /'\{\{([\s\S]*?)\}\}'/, contents = contents.replace(/('{{(.*)}}')/g, (substring: string, ...matches: string[]) => {
})({ if (!matches || !matches[1]) {
file: (path: string) => JSON.stringify(readFileSync(path, 'utf8')), return substring
version, }
const [replace, file] = matches[1].split(':')
if (replace !== 'replace') {
return substring
}
console.log('Replacing: ', substring)
return replacements[+file] ? replacements[+file] : JSON.stringify(readFileSync(file, 'utf8'))
}) })
writeFileSync(filename, contents) writeFileSync(filename, contents)
console.log(`Wrote: ${filename}`) console.log(`Wrote: ${filename}`)
-10
View File
@@ -1,10 +0,0 @@
{
"extends": "../tsconfig.json",
"compilerOptions": {
"noEmit": true
},
"include":[
"*.ts"
]
}
-23
View File
@@ -1,23 +0,0 @@
const { mkdtemp, copyFile } = require('fs/promises')
const os = require('os')
const path = require('path')
const rimraf = require('rimraf')
const cp = require('child_process')
async function runTests() {
const tempdir = await mkdtemp(path.join(os.tmpdir(), 'nexe-integration-tests-'))
const executable = path.join(tempdir, path.basename(process.argv[0]))
await copyFile(process.argv[0], executable)
process.on('beforeExit', () => rimraf.sync(tempdir))
cp.spawn(executable,
[
path.join(tempdir, 'node_modules/mocha/bin/mocha.js'),
path.join(tempdir, 'test/integration/tests.integration-spec.js')
],
{ stdio: ['inherit', 'inherit', 'inherit', 'ipc'], cwd: tempdir }
).on('exit', (code) => {
process.exitCode = code
})
}
runTests()
-120
View File
@@ -1,120 +0,0 @@
const { expect } = require('chai')
const fs = require('fs')
const { dirname, basename } = require('path')
const files = { zipFs: 'test/integration/tests.integration-spec.js', real: 'real', real2: 'real2' }
describe('SnapshotZipFS', () => {
console.error('Note: these tests are order dependant, so subsequent failures are likely a flow on effect.')
it('checks existance from the zipFs', () => expect(fs.existsSync(files.zipFs)).to.equal(true))
it('checks non-existance from the zipFs', () => expect(fs.existsSync('not found')).to.equal(false))
it('reads from the zipFs', () => expect(fs.readFileSync(files.zipFs, 'utf8')).to.include('SnapshotZipFS'))
it('copies from the zipFs', () => expect(() => fs.copyFileSync(files.zipFs, files.real)).not.to.throw())
it('reads from the real fs', () => expect(fs.readFileSync(files.real, 'utf8')).to.include('SnapshotZipFS'))
it('checks non-existance from the real fs', () => expect(fs.existsSync(files.real2)).to.equal(false))
it('copies from the real fs', () => expect(() => fs.copyFileSync(files.real, files.real2)).not.to.throw())
it('checks existance from the real fs', () => expect(fs.existsSync(files.real2)).to.equal(true))
it('unlinks from the real fs', () => expect(() => fs.unlinkSync(files.real2)).not.to.throw())
it('symlinks the real fs', () => expect(() => fs.symlinkSync('tmp', files.real2)).not.to.throw())
it('readlinks the real fs', () => expect(fs.readlinkSync(files.real2)).to.equal('tmp'))
it('truncates the real fs', () => expect(() => fs.truncateSync(files.real)).not.to.throw())
it('writeFiles the real fs', () => expect(() => fs.writeFileSync(files.real2, 'testing')).not.to.throw())
it('accesses the real fs', () => expect(() => fs.accessSync(files.real)).not.to.throw())
it('accesses the zipFs', () => expect(() => fs.accessSync(files.zipFs)).not.to.throw())
it('stats the real fs', () => expect(fs.statSync(files.real2).size).to.be.greaterThan(0))
it('stats the zipFs', () => expect(fs.statSync(files.zipFs).size).to.be.greaterThan(0))
it('readdirs from the zipFs', () => expect(fs.readdirSync(dirname(files.zipFs))).to.include(basename(files.zipFs)))
it('readdirs from the real fs', () => expect(fs.readdirSync('.')).to.include(files.real))
it('readdirs from both', () => {
expect(fs.readdirSync('.')).to.include(files.real),
expect(fs.readdirSync('.')).to.include('test')
})
let dir
it('opendirs the zipFs fs', async () => {
expect(() => dir = fs.opendirSync(dirname(files.zipFs))).not.to.throw()
const names = []
for await (const dirent of dir) {
names.push(dirent.name)
}
expect(names).to.include(basename(files.zipFs))
})
it('opendirs the real fs and zipFs', async () => {
expect(() => dir = fs.opendirSync('.')).not.to.throw()
const names = []
for await (const dirent of dir) {
names.push(dirent.name)
}
expect(names).to.include(files.real)
expect(names).to.include('test')
})
it('createWriteStreams to the real fs', () => expect(() => fs.createWriteStream('createWriteStream')).not.to.throw())
it('createReadStreams to the real fs', () => expect(() => fs.createReadStream(files.real)).not.to.throw())
it('createReadStreams to the zipFs', () => expect(() => fs.createReadStream(files.zipFs)).not.to.throw())
it('renames to the realFs', () => {
expect(() => fs.renameSync(files.real2, 'rename')).not.to.throw()
expect(fs.existsSync('rename')).to.equal(true)
})
describe('write actions to the zipFs', () => {
it(`won't truncate to the zipFs`, () => expect(() => fs.truncateSync(files.zipFs)).to.throw())
it(`won't unlink from the zipFs`, () => expect(() => fs.unlinkSync(files.zipFs)).to.throw())
it(`won't appendFile to the zipFs`, () => expect(() => fs.appendFileSync(files.zipFs, 'xxx')).to.throw())
it(`won't symlink to the zipFs`, () => expect(() => fs.symlinkSync('tmp', files.zipFs)).to.throw())
it(`won't chmod to the zipFs`, () => expect(() => fs.chmodSync(files.zipFs, 0o666)).to.throw())
it(`won't chown to the zipFs`, () => expect(() => fs.chownSync(files.zipFs, 0, 0)).to.throw())
it(`won't writeFile the zipFs`, () => expect(() => fs.writeFileSync(files.zipFs, 'testing')).to.throw())
it(`won't createWriteStreams to the zipFs`, () => expect(() => fs.createWriteStream(files.zipFs)).to.throw())
it(`won't rename to the zipFs`, () => expect(() => fs.renameSync(files.real, files.zipFs)).to.throw())
})
;[ 'zipFs', 'real' ].forEach((k) => {
it(`watches the ${k} fs`, () => expect(() => fs.watch(files[k], () => null).close()).not.to.throw())
it(`watchFiles the ${k} fs`, () => expect(() => fs.watchFile(files[k], () => null)).not.to.throw())
it(`unwatchFiles the ${k} fs`, () => expect(() => fs.unwatchFile(files[k])).not.to.throw())
})
let buffer = Buffer.alloc(8092), fd
describe('zipfs file descriptors', () => {
it('opens files in the zipFs', () => {
expect(() => (fd = fs.openSync(files.zipFs, 'a+'))).not.to.throw()
expect(fd).to.not.be.null
})
it('reads file descriptors from the zipFs', () => {
expect(() => fs.readSync(fd, buffer)).not.to.throw()
expect(buffer.toString()).to.include('SnapshotZipFS')
})
it('fstats the zipFs', () => expect(fs.fstatSync(fd).size).to.be.greaterThan(0))
it(`won't write to the zipFs`, () => expect(() => fs.writeSync(fd, 'xxx')).to.throw())
it(`won't fchmod the zipFs`, () => expect(() => fs.fchmodSync(fd)).to.throw())
it(`won't fchown the zipFs`, () => expect(() => fs.fchownSync(fd)).to.throw())
it(`won't ftruncate the zipFs`, () => expect(() => fs.ftruncateSync(fd)).to.throw())
it('closes file descriptors from the zipFs', () => expect(() => fs.closeSync(fd)).not.to.throw())
})
fd = null
buffer = Buffer.alloc(8092)
describe('real file descriptors', () => {
it('opens files in the real fs', () => {
expect(() => (fd = fs.openSync(files.real2, 'a+'))).not.to.throw()
expect(fd).to.not.be.null
})
it('reads file descriptors from the real fs', () => {
expect(() => fs.readSync(fd, buffer)).not.to.throw()
expect(buffer.toString()).to.include('SnapshotZipFS')
})
it('will write to the real fs', () => expect(() => fs.writeSync(fd, 'xxx')).not.to.throw())
it('fstats the real fs', () => expect(fs.fstatSync(fd).size).to.be.greaterThan(0))
it('closes file descriptors from the real fs', () => expect(() => fs.closeSync(fd)).not.to.throw())
})
describe('path clashes', () => {
it('delegates directory creation to the real fs', () => expect(() => fs.mkdirSync('test')).not.to.throw())
it('delegates directory removal to the real fs', () => expect(() => fs.rmdirSync('test')).not.to.throw())
})
describe('requiring modules', () => {
it('resolves modules from the zipFs', () => expect(() => require('resolve-dependencies')).not.to.throw())
it('resolves modules from the real fs', () => {
fs.mkdirSync('node_modules')
fs.mkdirSync('node_modules/fake-package')
fs.writeFileSync('node_modules/fake-package/package.json', '{ "name": "fake-package", "main": "some-file.js"}')
fs.writeFileSync('node_modules/fake-package/some-file.js', `module.exports = { fake: 'package' }`)
expect(require('fake-package')).to.eql({ fake: 'package' })
})
})
})
+4
View File
@@ -0,0 +1,4 @@
./test/**/*.spec.ts
--check-leaks
--require ts-node/register
--recursive
-20
View File
@@ -39,26 +39,6 @@ describe('options', () => {
expect(options.output).to.equal(path.resolve(cwd, `abc${ext}`)) expect(options.output).to.equal(path.resolve(cwd, `abc${ext}`))
}) })
}) })
describe('remote', () => {
it('should use default remote', () => {
const options = normalizeOptions({})
expect(options.remote).to.equal('https://github.com/nexe/nexe/releases/download/v3.3.3/')
})
it('should append trailing slash to third-party remote if necessary', () => {
const options = normalizeOptions({
remote: 'https://sitejs.org/nexe'
})
expect(options.remote).to.equal('https://sitejs.org/nexe/')
})
it('should not append trailing slash to third-party remote that already has one', () => {
const options = normalizeOptions({
remote: 'https://sitejs.org/nexe/'
})
expect(options.remote).to.equal('https://sitejs.org/nexe/')
})
})
describe('output', () => { describe('output', () => {
it('should work', () => { it('should work', () => {
const options = normalizeOptions({ const options = normalizeOptions({
+1 -2
View File
@@ -1,9 +1,8 @@
{ {
"compilerOptions": { "compilerOptions": {
"target":"es2015", "target":"es5",
"skipLibCheck": true, "skipLibCheck": true,
"experimentalDecorators": true, "experimentalDecorators": true,
"esModuleInterop": true,
"lib": ["es2018"], "lib": ["es2018"],
"module": "commonjs", "module": "commonjs",
"outDir": "./lib", "outDir": "./lib",
-7
View File
@@ -1,7 +0,0 @@
import * as wcp from 'webpack-config-prefabs'
export = wcp.nodeLibrary(module, {
enableTypescript: true,
entry: './src/fs/patch.ts',
minimize: false,
outputFilepath: './lib/fs/patch.bundle.js',
});