commit fe29170b4ea2a92ff5dd3bb6bc514b51147b71ef
Author: Greg Althaus <galthaus@austin.rr.com>
Date: Sat Jun 26 15:05:31 2021 -0500
build: update to go 1.16
M .travis.yml
M tools/test.sh
commit 72b21cc3220c6b39041b3b5c27a3cc9423727c40
Author: Victor Lowther <victor.lowther@gmail.com>
Date: Tue Jun 22 13:05:10 2021 -0500
perf(index): Track primary keys in the indexes
When dealing with large indexes, having to recalculate Key() quickly
adds a significant amount CPU and memory pressure for several object
categories that also make our largest indexes.
To alleviate this CPU and memory pressure, have the indexes also keep
track of their primary keys, and use them wherever it is reasonable
instead of recalculating Key() over and over.
Along the way, fix a few other issues:
* Optimize object removal from indexes both in the request tracker and
in the main index code.
* Don't make deep copies of objects we just deleted when publishing
events.
* Add dedicated Counts function to RequestTracker, and modify the
frontend to use it. This path avoids taking locks on each
individual object being listed.
* Content pack loading will now notice when you trye to load an object
that has no primary key, and complain mightly about it.
* Turned a couple more paired mutex lock/unlock operations into
thunking function.
* Fix a few reporting bugs in the bulk job purge reporting code.
* Fix testing-specific handling of ABRT
M backend/dataTracker.go
M backend/endpoint_manager.go
M backend/machines.go
M backend/requestTracker.go
M clitest/test-data/bob2.yaml
M clitest/test-data/output/TestContentLoadValidation/contents.upload.test-data/bob2.yaml/stderr.expect
M commit/replayer.go
M consensus/persistStore.go
M consensus/raftFSM.go
M consensus/serverAPI.go
M datastack/content.go
M datastack/index/index.go
M datastack/index/index_test.go
M datastack/index/priorityMerge.go
M datastack/layer.go
M datastack/stack.go
M frontend/frontend.go
M frontend/jobs.go
M frontend/pools.go
M server/args.go
End of Note