commit 8c1ea83433b85ad13470489a7df46fdde46ba996
Author: Greg Althaus <galthaus@austin.rr.com>
Date: Mon Oct 25 13:30:22 2021 -0500
feat(jobs): add a pref that will delay job purges until idle for that amount of time
This removes all job purges from directly inline with job creation.
M backend/dataTracker.go
M backend/jobs.go
M backend/machines.go
M clitest/test-data/output/TestContentCli/contents.list.2/stdout.expect
M clitest/test-data/output/TestContentCli/contents.list.3/stdout.expect
M clitest/test-data/output/TestContentCli/contents.list.4/stdout.expect
M clitest/test-data/output/TestContentCli/contents.list.5/stdout.expect
M clitest/test-data/output/TestContentCli/contents.list/stdout.expect
M clitest/test-data/output/TestContentsFunctionalCli/contents.list.2/stdout.expect
M clitest/test-data/output/TestContentsFunctionalCli/contents.list/stdout.expect
M frontend/job_create.go
M server/args.go
commit 951b38237f951467b3e30e9342861f5691f5b203
Author: Greg Althaus <galthaus@austin.rr.com>
Date: Mon Oct 25 11:58:31 2021 -0500
fix(raft): Test fix for reuse of certs
M consensus/raft_test.go
commit 52f9b3c85846d980b940b4a84383060077828866
Author: Victor Lowther <victor.lowther@gmail.com>
Date: Mon Oct 25 14:50:44 2021 -0500
Derp, fix silly test errors
M clitest/00_aone_test.go
M clitest/test-data/output/TestPluginProviderLockout/plugin_providers.upload.dangerzone.from/tools/build/dangerzone.2/stdout.expect
M clitest/test-data/output/TestPluginProviderLockout/plugin_providers.upload.incrementer.from/tools/build/incrementer.2/stdout.expect
commit 854204cb0b55eaf2004178911275854aa59e0208
Author: Victor Lowther <victor.lowther@gmail.com>
Date: Mon Oct 25 14:10:26 2021 -0500
feat(contents): Add contentsLocked preference.
You can use this new preference to lock out any API driven content
changes. Useful if you need to make sure no-one can change contents
they should not, even if they have the rights to do so.
M backend/dataTracker.go
M clitest/00_aone_test.go
M clitest/contents_test.go
M clitest/test-data/output/TestContentCli/contents.list.2/stdout.expect
M clitest/test-data/output/TestContentCli/contents.list.3/stdout.expect
M clitest/test-data/output/TestContentCli/contents.list.4/stdout.expect
M clitest/test-data/output/TestContentCli/contents.list.5/stdout.expect
M clitest/test-data/output/TestContentCli/contents.list/stdout.expect
M clitest/test-data/output/TestContentsFunctionalCli/contents.list.2/stdout.expect
M clitest/test-data/output/TestContentsFunctionalCli/contents.list/stdout.expect
A clitest/test-data/output/TestPluginProviderLockout/contents.destroy.john.2/stdout.expect
A clitest/test-data/output/TestPluginProviderLockout/contents.destroy.john/stderr.expect
A clitest/test-data/output/TestPluginProviderLockout/contents.upload.4c4e8ae2275d5c4ad58d3e7b5442a889.2/stdout.expect
A clitest/test-data/output/TestPluginProviderLockout/contents.upload.4c4e8ae2275d5c4ad58d3e7b5442a889/stderr.expect
A clitest/test-data/output/TestPluginProviderLockout/plugin_providers.destroy.dangerzone.2/stdout.expect
A clitest/test-data/output/TestPluginProviderLockout/plugin_providers.destroy.dangerzone/stderr.expect
A clitest/test-data/output/TestPluginProviderLockout/plugin_providers.destroy.incrementer.2/stdout.expect
A clitest/test-data/output/TestPluginProviderLockout/plugin_providers.destroy.incrementer/stderr.expect
A clitest/test-data/output/TestPluginProviderLockout/plugin_providers.upload.dangerzone.from/tools/build/dangerzone.2/stdout.expect
A clitest/test-data/output/TestPluginProviderLockout/plugin_providers.upload.dangerzone.from/tools/build/dangerzone/stderr.expect
A clitest/test-data/output/TestPluginProviderLockout/plugin_providers.upload.incrementer.from/tools/build/incrementer.2/stdout.expect
A clitest/test-data/output/TestPluginProviderLockout/plugin_providers.upload.incrementer.from/tools/build/incrementer/stderr.expect
A clitest/test-data/output/TestPluginProviderLockout/prefs.set.contentLocked.false.2/stdout.expect
A clitest/test-data/output/TestPluginProviderLockout/prefs.set.contentLocked.false/stdout.expect
A clitest/test-data/output/TestPluginProviderLockout/prefs.set.contentLocked.true.2/stdout.expect
A clitest/test-data/output/TestPluginProviderLockout/prefs.set.contentLocked.true/stdout.expect
M frontend/content.go
M frontend/plugin_providers.go
commit b2fb136341a973c3e69316f21660e1da966645da
Author: Victor Lowther <victor.lowther@gmail.com>
Date: Mon Oct 25 12:01:42 2021 -0500
Stream returing a single content bundle like we would any other object
M frontend/content.go
commit 606bdb170df70d9ea1b06bd2e7f18647ce214918
Author: Victor Lowther <victor.lowther@gmail.com>
Date: Mon Oct 25 11:58:51 2021 -0500
Locking everything to get content summaries is not actually needed
M frontend/content.go
commit 10b52f499d812ffc44b1206a25134bd130cdd009
Author: Victor Lowther <victor.lowther@gmail.com>
Date: Mon Oct 25 11:51:17 2021 -0500
Skip reflect conversion compatibility checking when all we care about is the actual underlying byte slice
M datastack/index/index.go
commit 16a28f1d9ebd97778d0ba47bc42cd16e59292651
Author: Victor Lowther <victor.lowther@gmail.com>
Date: Mon Oct 25 11:50:11 2021 -0500
Skip machine uniqieness checking if hte name did not change
M backend/machines.go
commit 5a97e10c2c7f85b3e5cdeeb3720382c042f97df2
Author: Victor Lowther <victor.lowther@gmail.com>
Date: Mon Oct 25 11:29:25 2021 -0500
Have generic frontend.Remove code use encodeItemTo
M frontend/frontend.go
commit dd1201592ca62a14ab82937674f2f35fdf358d85
Author: Victor Lowther <victor.lowther@gmail.com>
Date: Mon Oct 25 11:28:20 2021 -0500
Relax models.Models restriction on generic frontend encoding stuff
M frontend/frontend.go
M mr/modelReflect.go
commit 36dd1b582d505b172936facd648b4e388c766002
Author: Victor Lowther <victor.lowther@gmail.com>
Date: Mon Oct 25 11:26:09 2021 -0500
Add perf test for messing with a million -ish jobs
M perf_tests/machineCreate_test.go
commit 93ce7e82d2b3435dc7e054575c625bb82bba2426
Author: Victor Lowther <victor.lowther@gmail.com>
Date: Thu Oct 21 14:31:41 2021 -0500
perf(frontend): Optimize frontend memory usage.
This refactors various frontend functions to reduce memory consumption
in a few different ways:
1. Add frontend.Resolve and frontend.RawFind methods. The former is
optimized for first-pass object availability testing, and the second
is frontend.Find refactored to operate inside a transaction where the
expectation is that the object fetched will not be modified or sent
over the wire by the end of the API call.
2. Replace frontend.processItem with a collection of functions that
let us interleave JSON encoding and any required mangling on objects
we need to send over the API (for param decoding, object sanitization,
slimming, etc). It utilizes a combination of reflection and unwinding
the top layer of JSON encoding to let us make field-by-field decisions
on whether to skip or mangle arbitrary object fields. Use this new
functionality for the main read and write paths through the API.
3. Sending Events no longer carries around deep copies of the objects
the event is being sent for. Instead, we marshal them beforehand and
arrange for the marshalled data to be slotted in directly.
4. Standardize on the jsoniter codec object that all the packages use
and get rid of some legacy use of encoding/json in the front end.
On my dev box, this reduces the time spent in the machine create and
list performance test from 40.07 seconds -> 15.47 seconds in the API,
and (more importantly) 12.45 seconds -> 3.72 seconds in the garbage
collector.
M backend/dataTracker.go
M backend/machines.go
M backend/requestTracker.go
M backend/utils.go
M clitest/common_test.go
M clitest/test-data/output/TestPluginProviderCli/plugin_providers.list.2/stdout.expect
M clitest/test-data/output/TestSecureParams/91e88b16a9c5b7e77215714ef9eba6ff/stderr.expect
M clitest/test-data/output/TestSecureParams/bb5d4e3f789d07bec23bc261352ea358/stderr.expect
M clitest/test-data/output/TestSecureParams/e19676fa6a1f473aaa7f7124105765d8/stderr.expect
M commit/buffer.go
M commit/entry.go
M commit/entry_easyjson.go
M frontend/actions.go
M frontend/bootenvs.go
M frontend/endpoint.go
M frontend/endpoints.go
M frontend/frontend.go
M frontend/jobs.go
M frontend/meta.go
M frontend/params.go
M frontend/pools.go
M frontend/users.go
M frontend/websocket.go
A mr/modelReflect.go
A utils/json.jo.go
commit 359e9076cf920e2cb486e0327ffca7fcccdb1f03
Author: Victor Lowther <victor.lowther@gmail.com>
Date: Sun Oct 24 13:20:51 2021 -0500
perf(frontend): Start adding some basic frontend perf testing.
So far, just single-threaded machines create. To use this:
1. cd perf_tests
2. go generate
3. go test -cpuprofile cpu.out.before
4. hackety hack hack for great perf enhancements
5. go test -cpuprofile cpu.out.after
Then, marvel at your magnificence. You can also use -memprofile
and/or -trace to better optimize other things.
Other perf testy things will also appear here.
M .gitignore
M cmds/license/license.go
A cmds/license/machinePerf.go
A perf_tests/machineCreate_test.go
A perf_tests/main_test.go
M tools/test.sh
End of Note