Commit graph

67 commits

Author SHA1 Message Date
Ryan Huffman
c2f08dfdba Update Octree to atomically persist data 2018-06-13 12:24:09 -07:00
Ryan Huffman
6b159d6124 Remove redundant read of octree data on startup 2018-06-13 11:54:20 -07:00
Ryan Huffman
fa22ea7630 Update ES to keep the backup that pushes file size over the limit 2018-06-11 11:58:50 -07:00
Ryan Huffman
11a48915e2 Fix order of variable initialization in OctreePersistThread 2018-06-11 09:48:22 -07:00
Ryan Huffman
94069c964d Remove unused variable and log message in OctreePersistThread 2018-06-11 08:37:19 -07:00
Ryan Huffman
9afc9d19c5 Cleanup comments and process structure in OctreePersistThread 2018-06-08 16:22:11 -07:00
Ryan Huffman
d0af06939f Update persistInterval to use std::chrono in OctreePersistThread 2018-06-08 16:22:11 -07:00
Ryan Huffman
8f4f3db9ce Add octree backup file cleanup 2018-06-08 16:22:11 -07:00
Ryan Huffman
d476146a56 Move OctreePersistThread off GenericThread and add octree negotiation 2018-06-08 16:22:11 -07:00
Ryan Huffman
2143bae100 Remove unused entity server backups
The domain server now handles entity backups so this code is no longer
used.

This also fixes a bug where, if the entity server went down while
writing the persist file, it would leave a lock file behind. When
starting up the next time it would try to recover from the last backup.
Because the entity server no longer gets any backup rules, and the way
it searches for backups, it would end up finding the lock file and
treating it as the most recent backup. This meant that the
models.json.gz file was replaced with an empty file.
2018-06-08 16:17:30 -07:00
David Back
ac33f33828 update version from json in toByteArray 2018-05-31 09:24:16 -07:00
Clement
da9fb9c751 Fix non self-sufficient headers 2018-05-04 16:26:41 -07:00
Ryan Huffman
010356709d Remove lock file recovery in entity server
* We no longer make backups at the ES, so there is no backup to
  recover from if we crash during persist to disk.
  * When loading the most recent backup we look for files that match our
  backup rules. If we have no backup rules we end up grabbing
  everything.
  * Because we grab everything, we are at risk of grabbing the lock file
  and treating it as a backup. In the case where we crash on persist, we
  end up replacing the models.json.gz file with the lock file (!!).

This commit makes a small change to not do the recovery on startup. In a
future update we will remove the backup-related code on the ES
altogether.
2018-04-05 11:52:31 -07:00
Ryan Huffman
70bdedc4b3 Fix Edit -> Export Entities exporting gzipped json
We were not respecting the doGzip flag.
2018-03-01 16:42:41 -08:00
Ryan Huffman
ec210e1750 Move OctreeUtils to OctreeDataUtils 2018-02-26 16:51:42 -08:00
Ryan Huffman
3fa538d42d Add RawEntityData subclass 2018-02-26 16:32:09 -08:00
Ryan Huffman
2a667fcd60 Cleanup entity -> ds persist 2018-02-15 10:09:21 -08:00
Ryan Huffman
ff5be2d690 Fix entity data ID sometimes being reset 2018-02-15 10:09:20 -08:00
Ryan Huffman
cb9327e030 Add entity file sync and domain content backups 2018-02-15 10:09:20 -08:00
Stephen Birarda
9bc0609b1e address code review comments 2017-04-28 10:17:48 -07:00
Stephen Birarda
a21a34a4a4 add entities file replacement to DS and ES 2017-04-27 16:56:27 -07:00
Ryan Huffman
727d59ab27 Add backup directory setting to domain server 2016-08-16 17:33:32 -07:00
Brad Hefta-Gaub
fe14bc8e52 also switch OctreePersistThread to use std::this_thread::sleep_for() 2016-05-06 11:19:27 -07:00
Stephen Birarda
7dbf29821a don't persist the octree if it never loaded 2015-12-01 10:00:25 -08:00
Brad Hefta-Gaub
456da661ff implement support to download the persist file from entity server status page 2015-11-18 17:30:07 -08:00
Brad Davis
9b81842850 Merge branch 'master' into homer
Conflicts:
	assignment-client/src/octree/OctreeSendThread.cpp
	interface/src/Application.cpp
	libraries/entities-renderer/src/EntityTreeRenderer.cpp
	libraries/entities/src/EntityItem.cpp
	libraries/entities/src/EntityTree.cpp
	libraries/entities/src/EntityTreeElement.cpp
	libraries/entities/src/EntityTreeElement.h
	libraries/entities/src/EntityTreeHeadlessViewer.cpp
	libraries/entities/src/RecurseOctreeToMapOperator.cpp
	libraries/octree/src/Octree.h
	libraries/physics/src/EntityMotionState.cpp
2015-09-11 16:31:18 -07:00
Brad Davis
aa989c3687 Fixing some entity race condition crashes 2015-09-10 20:53:17 -07:00
Seth Alves
31fcd55d47 octree uses shared pointers 2015-09-08 11:59:48 -07:00
David Rowe
709508857c Delete maximum rolling backup file if it exists before rolling backup 2015-06-18 17:39:31 -07:00
Stephen Birarda
edade7baba resolve conflicts on merge with upstream master 2015-05-01 16:24:16 -07:00
Stephen Birarda
94739cf8d3 move number constants out of SharedUtil to NumericalConstants 2015-05-01 10:26:53 -07:00
Seth Alves
d2cd4fc405 types 0 and 1 exit cleanly, type 6 crashes 2015-04-29 11:56:45 -07:00
ZappoMan
512a109023 add call to perror() to get OS level error details on failed backups 2015-04-28 13:48:51 -07:00
Seth Alves
91861e9e5a split out octree logging 2015-04-06 17:04:49 -07:00
Seth Alves
bed37ef49a incorporate code review feedback 2015-03-16 13:57:22 -07:00
Seth Alves
9e6c289ab8 move filename utils into shared library 2015-03-12 10:23:57 -07:00
Seth Alves
3bc85c2531 instead of a json checkbox, a dropdown to chose the file type 2015-03-12 09:34:01 -07:00
Seth Alves
ac882f86e2 backup code is now aware of possible file extension differences 2015-03-12 09:13:53 -07:00
Seth Alves
2a06816f71 added persist-as-json checkbox to domain settings. File extension of persist file is automatically adjusted when entity server saves. When loading, if both a json and svo file exist, the newer one is used. This is regardless of the persistAsJson setting 2015-03-11 23:09:15 -07:00
Seth Alves
2076de9153 started on reading entities from json file 2015-03-10 10:12:29 -07:00
Seth Alves
81dc7cb8cf started on code to persist entities to file as json rather than in the wire protocol 2015-03-09 13:38:18 -07:00
ZappoMan
573d7971ac fix a couple of bugs in backup settings 2015-02-13 20:23:12 -08:00
ZappoMan
f1a5865201 added more logging to persist and backup, fixed bug in start up case for no models.svo and backup 2015-02-12 15:17:48 -08:00
ZappoMan
0e19e1f8a5 cleanup some dead code 2015-01-13 13:48:28 -08:00
ZappoMan
60aec8ac00 add support for multiple backup rules 2015-01-13 13:41:32 -08:00
ZappoMan
ea3a5b7fcd use last backup files timestamp as last backup time. 2015-01-12 17:29:10 -08:00
ZappoMan
7b65c75414 first cut at restoreFromMostRecentBackup 2015-01-12 14:32:13 -08:00
ZappoMan
1562aab6da intitial pass at lock file 2015-01-12 12:35:17 -08:00
ZappoMan
17ed92cf7e fix runaway backup bug 2014-12-23 10:11:08 -08:00
ZappoMan
bdfe02bbd0 Merge branch 'master' of https://github.com/highfidelity/hifi into persistThreadImprovements
Conflicts:
	assignment-client/src/octree/OctreeServer.cpp
	libraries/octree/src/OctreePersistThread.cpp
	libraries/octree/src/OctreePersistThread.h
2014-11-17 11:22:00 -08:00