diff --git a/BUILD.md b/BUILD.md index 5abb3ae4e7..c1ccd3193e 100644 --- a/BUILD.md +++ b/BUILD.md @@ -1,7 +1,7 @@ ###Dependencies -* [cmake](http://www.cmake.org/cmake/resources/software.html) ~> 2.8.12.2 -* [Qt](http://www.qt.io/download-open-source) ~> 5.4.1 +* [cmake](http://www.cmake.org/cmake/resources/software.html) ~> 3.3.2 +* [Qt](http://www.qt.io/download-open-source) ~> 5.5.1 * [OpenSSL](https://www.openssl.org/community/binaries.html) ~> 1.0.1m * IMPORTANT: Using the recommended version of OpenSSL is critical to avoid security vulnerabilities. * [VHACD](https://github.com/virneo/v-hacd)(clone this repository)(Optional) @@ -21,10 +21,10 @@ * [SDL2](https://www.libsdl.org/download-2.0.php) ~> 2.0.3 * [soxr](http://soxr.sourceforge.net) ~> 0.1.1 * [Intel Threading Building Blocks](https://www.threadingbuildingblocks.org/) ~> 4.3 -* [Sixense](http://sixense.com/) ~> 071615 +* [Sixense](http://sixense.com/) ~> 071615 * [zlib](http://www.zlib.net/) ~> 1.28 (Win32 only) -The above dependencies will be downloaded, built, linked and included automatically by CMake where we require them. The CMakeLists files that handle grabbing each of the following external dependencies can be found in the [cmake/externals folder](cmake/externals). The resulting downloads, source files and binaries will be placed in the `build/ext` folder in each of the subfolders for each external project. +The above dependencies will be downloaded, built, linked and included automatically by CMake where we require them. The CMakeLists files that handle grabbing each of the following external dependencies can be found in the [cmake/externals folder](cmake/externals). The resulting downloads, source files and binaries will be placed in the `build/ext` folder in each of the subfolders for each external project. These are not placed in your normal build tree when doing an out of source build so that they do not need to be re-downloaded and re-compiled every time the CMake build folder is cleared. Should you want to force a re-download and re-compile of a specific external, you can simply remove that directory from the appropriate subfolder in `build/ext`. Should you want to force a re-download and re-compile of all externals, just remove the `build/ext` folder. @@ -42,12 +42,12 @@ Hifi uses CMake to generate build files and project files for your platform. ####Qt In order for CMake to find the Qt5 find modules, you will need to set an ENV variable pointing to your Qt installation. -For example, a Qt5 5.4.1 installation to /usr/local/qt5 would require that QT_CMAKE_PREFIX_PATH be set with the following command. This can either be entered directly into your shell session before you build or in your shell profile (e.g.: ~/.bash_profile, ~/.bashrc, ~/.zshrc - this depends on your shell and environment). +For example, a Qt5 5.5.1 installation to /usr/local/qt5 would require that QT_CMAKE_PREFIX_PATH be set with the following command. This can either be entered directly into your shell session before you build or in your shell profile (e.g.: ~/.bash_profile, ~/.bashrc, ~/.zshrc - this depends on your shell and environment). The path it needs to be set to will depend on where and how Qt5 was installed. e.g. - export QT_CMAKE_PREFIX_PATH=/usr/local/qt/5.4.1/clang_64/lib/cmake/ - export QT_CMAKE_PREFIX_PATH=/usr/local/Cellar/qt5/5.4.1/lib/cmake + export QT_CMAKE_PREFIX_PATH=/usr/local/qt/5.5.1/clang_64/lib/cmake/ + export QT_CMAKE_PREFIX_PATH=/usr/local/Cellar/qt5/5.5.1/lib/cmake export QT_CMAKE_PREFIX_PATH=/usr/local/opt/qt5/lib/cmake ####Generating build files @@ -64,7 +64,7 @@ Any variables that need to be set for CMake to find dependencies can be set as E For example, to pass the QT_CMAKE_PREFIX_PATH variable during build file generation: - cmake .. -DQT_CMAKE_PREFIX_PATH=/usr/local/qt/5.4.1/lib/cmake + cmake .. -DQT_CMAKE_PREFIX_PATH=/usr/local/qt/5.5.1/lib/cmake ####Finding Dependencies @@ -83,4 +83,3 @@ In the examples below the variable $NAME would be replaced by the name of the de ####Devices You can support external input/output devices such as Leap Motion, MIDI, and more by adding each individual SDK in the visible building path. Refer to the readme file available in each device folder in [interface/external/](interface/external) for the detailed explanation of the requirements to use the device. - diff --git a/BUILD_ANDROID.md b/BUILD_ANDROID.md index 9f86e7e925..88294f3040 100644 --- a/BUILD_ANDROID.md +++ b/BUILD_ANDROID.md @@ -14,7 +14,7 @@ You will need the following tools to build our Android targets. * Install the latest Platform-tools * Install the latest Build-tools * Install the SDK Platform for API Level 19 - * Install Sources for Android SDK for API Level 19 + * Install Sources for Android SDK for API Level 19 * Install the ARM EABI v7a System Image if you want to run an emulator. You will also need to cross-compile the dependencies required for all platforms for Android, and help CMake find these compiled libraries on your machine. @@ -25,7 +25,7 @@ You will also need to cross-compile the dependencies required for all platforms ####ANDROID_LIB_DIR -Since you won't be installing Android dependencies to system paths on your development machine, CMake will need a little help tracking down your Android dependencies. +Since you won't be installing Android dependencies to system paths on your development machine, CMake will need a little help tracking down your Android dependencies. This is most easily accomplished by installing all Android dependencies in the same folder. You can place this folder wherever you like on your machine. In this build guide and across our CMakeLists files this folder is referred to as `ANDROID_LIB_DIR`. You can set `ANDROID_LIB_DIR` in your environment or by passing when you run CMake. @@ -45,7 +45,7 @@ The original instructions to compile OpenSSL for Android from your host environm Download the [OpenSSL source](https://www.openssl.org/source/) and extract the tarball inside your `ANDROID_LIB_DIR`. Rename the extracted folder to `openssl`. -You will need the [setenv-android.sh script](http://wiki.openssl.org/index.php/File:Setenv-android.sh) from the OpenSSL wiki. +You will need the [setenv-android.sh script](http://wiki.openssl.org/index.php/File:Setenv-android.sh) from the OpenSSL wiki. You must change three values at the top of the `setenv-android.sh` script - `_ANDROID_NDK`, `_ANDROID_EABI` and `_ANDROID_API`. `_ANDROID_NDK` should be `android-ndk-r10`, `_ANDROID_EABI` should be `arm-linux-androidebi-4.9` and `_ANDROID_API` should be `19`. @@ -62,8 +62,8 @@ source setenv-android.sh Then, from the OpenSSL directory, run the following commands. ``` -perl -pi -e 's/install: all install_docs install_sw/install: install_docs install_sw/g' Makefile.org -./config shared -no-ssl2 -no-ssl3 -no-comp -no-hw -no-engine --openssldir=/usr/local/ssl/$ANDROID_API +perl -pi -e 's/install: all install_docs install_sw/install: install_docs install_sw/g' Makefile.org +./config shared -no-ssl2 -no-ssl3 -no-comp -no-hw -no-engine --openssldir=/usr/local/ssl/$ANDROID_API make depend make all ``` @@ -78,7 +78,7 @@ The Oculus Mobile SDK is optional, for Gear VR support. It is not required to co Download the [Oculus Mobile SDK](https://developer.oculus.com/downloads/#sdk=mobile) and extract the archive inside your `ANDROID_LIB_DIR` folder. Rename the extracted folder to `libovr`. -From the VRLib directory, use ndk-build to build VrLib. +From the VRLib directory, use ndk-build to build VrLib. ``` cd VRLib @@ -107,4 +107,4 @@ The following must be set in your environment: The following must be passed to CMake when it is run: -* USE_ANDROID_TOOLCHAIN - set to true to build for Android \ No newline at end of file +* USE_ANDROID_TOOLCHAIN - set to true to build for Android diff --git a/BUILD_OSX.md b/BUILD_OSX.md index 9d1357d672..54360ad4b8 100644 --- a/BUILD_OSX.md +++ b/BUILD_OSX.md @@ -3,20 +3,15 @@ Please read the [general build guide](BUILD.md) for information on dependencies ###Homebrew [Homebrew](http://brew.sh/) is an excellent package manager for OS X. It makes install of all High Fidelity dependencies very simple. - brew tap highfidelity/homebrew-formulas - brew install cmake openssl - brew install highfidelity/formulas/qt5 - brew link qt5 --force + brew install cmake openssl qt5 -We have a [homebrew formulas repository](https://github.com/highfidelity/homebrew-formulas) that you can use/tap to install some of the dependencies. In the code block above qt5 is installed from a formula in this repository. - -*Our [qt5 homebrew formula](https://raw.github.com/highfidelity/homebrew-formulas/master/qt5.rb) is for a patched version of Qt 5.4.x stable that removes wireless network scanning that can reduce real-time audio performance. We recommended you use this formula to install Qt.* +We no longer require install of qt5 via our [homebrew formulas repository](https://github.com/highfidelity/homebrew-formulas). Versions of Qt that are 5.5.x and above provide a mechanism to disable the wireless scanning we previously had a custom patch for. ###Qt -Assuming you've installed Qt 5 using the homebrew instructions above, you'll need to set QT_CMAKE_PREFIX_PATH so CMake can find your installation of Qt. For Qt 5.4.1 installed via homebrew, set QT_CMAKE_PREFIX_PATH as follows. +Assuming you've installed Qt 5 using the homebrew instructions above, you'll need to set QT_CMAKE_PREFIX_PATH so CMake can find your installation of Qt. For Qt 5.5.1 installed via homebrew, set QT_CMAKE_PREFIX_PATH as follows. - export QT_CMAKE_PREFIX_PATH=/usr/local/Cellar/qt5/5.4.1/lib/cmake + export QT_CMAKE_PREFIX_PATH=/usr/local/Cellar/qt5/5.5.1/lib/cmake ###Xcode If Xcode is your editor of choice, you can ask CMake to generate Xcode project files instead of Unix Makefiles. diff --git a/BUILD_WIN.md b/BUILD_WIN.md index 89646e99ff..48781ca34a 100644 --- a/BUILD_WIN.md +++ b/BUILD_WIN.md @@ -16,7 +16,7 @@ If using Visual Studio 2013 and building as a Visual Studio 2013 project you nee ####nmake -Some of the external projects may require nmake to compile and install. If it is not installed at the location listed below, please ensure that it is in your PATH so CMake can find it when required. +Some of the external projects may require nmake to compile and install. If it is not installed at the location listed below, please ensure that it is in your PATH so CMake can find it when required. We expect nmake.exe to be located at the following path. @@ -29,19 +29,19 @@ NOTE: Qt does not support 64-bit builds on Windows 7, so you must use the 32-bit * [Download the online installer](http://qt-project.org/downloads) * When it asks you to select components, ONLY select the following: - * Qt > Qt 5.4.1 > **msvc2013 32-bit OpenGL** + * Qt > Qt 5.5.1 > **msvc2013 32-bit** -* [Download the offline installer](http://download.qt.io/official_releases/qt/5.4/5.4.1/qt-opensource-windows-x86-msvc2013_opengl-5.4.1.exe) +* [Download the offline installer](http://download.qt.io/official_releases/qt/5.5/5.5.1/qt-opensource-windows-x86-msvc2013-5.5.1.exe) Once Qt is installed, you need to manually configure the following: -* Set the QT_CMAKE_PREFIX_PATH environment variable to your `Qt\5.4.1\msvc2013_opengl\lib\cmake` directory. +* Set the QT_CMAKE_PREFIX_PATH environment variable to your `Qt\5.5.1\msvc2013\lib\cmake` directory. * You can set an environment variable from Control Panel > System > Advanced System Settings > Environment Variables > New ###External Libraries As it stands, Hifi/Interface is a 32-bit application, so all libraries should also be 32-bit. -CMake will need to know where the headers and libraries for required external dependencies are. +CMake will need to know where the headers and libraries for required external dependencies are. We use CMake's `fixup_bundle` to find the DLLs all of our exectuable targets require, and then copy them beside the executable in a post-build step. If `fixup_bundle` is having problems finding a DLL, you can fix it manually on your end by adding the folder containing that DLL to your path. Let us know which DLL CMake had trouble finding, as it is possible a tweak to our CMake files is required. diff --git a/assignment-client/src/Agent.cpp b/assignment-client/src/Agent.cpp index 063bf24de8..1f56118177 100644 --- a/assignment-client/src/Agent.cpp +++ b/assignment-client/src/Agent.cpp @@ -17,6 +17,7 @@ #include #include +#include #include #include #include @@ -93,7 +94,7 @@ void Agent::handleJurisdictionPacket(QSharedPointer packet, SharedNode DependencyManager::get()->getJurisdictionListener()-> queueReceivedPacket(packet, senderNode); } -} +} void Agent::handleAudioPacket(QSharedPointer packet) { _receivedAudioStream.parseData(*packet); @@ -109,11 +110,21 @@ const int PING_INTERVAL = 1000; void Agent::run() { ThreadedAssignment::commonInit(AGENT_LOGGING_NAME, NodeType::Agent); + // Setup MessagesClient + auto messagesClient = DependencyManager::set(); + QThread* messagesThread = new QThread; + messagesThread->setObjectName("Messages Client Thread"); + messagesClient->moveToThread(messagesThread); + connect(messagesThread, &QThread::started, messagesClient.data(), &MessagesClient::init); + messagesThread->start(); + + auto nodeList = DependencyManager::get(); nodeList->addSetOfNodeTypesToNodeInterestSet(NodeSet() << NodeType::AudioMixer << NodeType::AvatarMixer << NodeType::EntityServer + << NodeType::MessagesMixer ); _pingTimer = new QTimer(this); diff --git a/assignment-client/src/AssignmentClient.cpp b/assignment-client/src/AssignmentClient.cpp index bf5f9c3b7f..2d11f4d289 100644 --- a/assignment-client/src/AssignmentClient.cpp +++ b/assignment-client/src/AssignmentClient.cpp @@ -198,7 +198,7 @@ void AssignmentClient::sendStatusPacketToACM() { } void AssignmentClient::sendAssignmentRequest() { - if (!_currentAssignment) { + if (!_currentAssignment && !_isAssigned) { auto nodeList = DependencyManager::get(); @@ -229,8 +229,9 @@ void AssignmentClient::handleCreateAssignmentPacket(QSharedPointer pac // construct the deployed assignment from the packet data _currentAssignment = AssignmentFactory::unpackAssignment(*packet); - if (_currentAssignment) { + if (_currentAssignment && !_isAssigned) { qDebug() << "Received an assignment -" << *_currentAssignment; + _isAssigned = true; auto nodeList = DependencyManager::get(); @@ -309,12 +310,11 @@ void AssignmentClient::handleAuthenticationRequest() { } void AssignmentClient::assignmentCompleted() { - // we expect that to be here the previous assignment has completely cleaned up assert(_currentAssignment.isNull()); - // reset our current assignment pointer to NULL now that it has been deleted - _currentAssignment = NULL; + // reset our current assignment pointer to null now that it has been deleted + _currentAssignment = nullptr; // reset the logging target to the the CHILD_TARGET_NAME LogHandler::getInstance().setTargetName(ASSIGNMENT_CLIENT_TARGET_NAME); @@ -330,4 +330,6 @@ void AssignmentClient::assignmentCompleted() { nodeList->setOwnerType(NodeType::Unassigned); nodeList->reset(); nodeList->resetNodeInterestSet(); + + _isAssigned = false; } diff --git a/assignment-client/src/AssignmentClient.h b/assignment-client/src/AssignmentClient.h index 9d2c816861..9d7591f931 100644 --- a/assignment-client/src/AssignmentClient.h +++ b/assignment-client/src/AssignmentClient.h @@ -46,6 +46,7 @@ private: Assignment _requestAssignment; QPointer _currentAssignment; + bool _isAssigned { false }; QString _assignmentServerHostname; HifiSockAddr _assignmentServerSocket; QTimer _requestTimer; // timer for requesting and assignment diff --git a/assignment-client/src/AssignmentFactory.cpp b/assignment-client/src/AssignmentFactory.cpp index cacc523ebd..c4cd6821ef 100644 --- a/assignment-client/src/AssignmentFactory.cpp +++ b/assignment-client/src/AssignmentFactory.cpp @@ -17,6 +17,7 @@ #include "avatars/AvatarMixer.h" #include "entities/EntityServer.h" #include "assets/AssetServer.h" +#include "messages/MessagesMixer.h" ThreadedAssignment* AssignmentFactory::unpackAssignment(NLPacket& packet) { @@ -36,6 +37,8 @@ ThreadedAssignment* AssignmentFactory::unpackAssignment(NLPacket& packet) { return new EntityServer(packet); case Assignment::AssetServerType: return new AssetServer(packet); + case Assignment::MessagesMixerType: + return new MessagesMixer(packet); default: return NULL; } diff --git a/assignment-client/src/entities/EntityServer.cpp b/assignment-client/src/entities/EntityServer.cpp index 5754a9e057..2fafaa6731 100644 --- a/assignment-client/src/entities/EntityServer.cpp +++ b/assignment-client/src/entities/EntityServer.cpp @@ -112,7 +112,6 @@ int EntityServer::sendSpecialPackets(const SharedNodePointer& node, OctreeQueryN quint64 deletePacketSentAt = usecTimestampNow(); EntityTreePointer tree = std::static_pointer_cast(_tree); auto recentlyDeleted = tree->getRecentlyDeletedEntityIDs(); - bool hasMoreToSend = true; packetsSent = 0; diff --git a/assignment-client/src/messages/MessagesMixer.cpp b/assignment-client/src/messages/MessagesMixer.cpp new file mode 100644 index 0000000000..d3662f3fb5 --- /dev/null +++ b/assignment-client/src/messages/MessagesMixer.cpp @@ -0,0 +1,134 @@ +// +// MessagesMixer.cpp +// assignment-client/src/messages +// +// Created by Brad hefta-Gaub on 11/16/2015. +// Copyright 2015 High Fidelity, Inc. +// +// Distributed under the Apache License, Version 2.0. +// See the accompanying file LICENSE or http://www.apache.org/licenses/LICENSE-2.0.html +// + +#include +#include +#include + +#include +#include +#include + +#include "MessagesMixer.h" + +const QString MESSAGES_MIXER_LOGGING_NAME = "messages-mixer"; + +MessagesMixer::MessagesMixer(NLPacket& packet) : + ThreadedAssignment(packet) +{ + // make sure we hear about node kills so we can tell the other nodes + connect(DependencyManager::get().data(), &NodeList::nodeKilled, this, &MessagesMixer::nodeKilled); + + auto& packetReceiver = DependencyManager::get()->getPacketReceiver(); + packetReceiver.registerMessageListener(PacketType::MessagesData, this, "handleMessages"); + packetReceiver.registerMessageListener(PacketType::MessagesSubscribe, this, "handleMessagesSubscribe"); + packetReceiver.registerMessageListener(PacketType::MessagesUnsubscribe, this, "handleMessagesUnsubscribe"); +} + +MessagesMixer::~MessagesMixer() { +} + +void MessagesMixer::nodeKilled(SharedNodePointer killedNode) { + for (auto& channel : _channelSubscribers) { + channel.remove(killedNode->getUUID()); + } +} + +void MessagesMixer::handleMessages(QSharedPointer packetList, SharedNodePointer senderNode) { + Q_ASSERT(packetList->getType() == PacketType::MessagesData); + + QByteArray packetData = packetList->getMessage(); + QBuffer packet{ &packetData }; + packet.open(QIODevice::ReadOnly); + + quint16 channelLength; + packet.read(reinterpret_cast(&channelLength), sizeof(channelLength)); + auto channelData = packet.read(channelLength); + QString channel = QString::fromUtf8(channelData); + + quint16 messageLength; + packet.read(reinterpret_cast(&messageLength), sizeof(messageLength)); + auto messageData = packet.read(messageLength); + QString message = QString::fromUtf8(messageData); + + auto nodeList = DependencyManager::get(); + + nodeList->eachMatchingNode( + [&](const SharedNodePointer& node)->bool { + + return node->getType() == NodeType::Agent && node->getActiveSocket() && + _channelSubscribers[channel].contains(node->getUUID()); + }, + [&](const SharedNodePointer& node) { + + auto packetList = NLPacketList::create(PacketType::MessagesData, QByteArray(), true, true); + + auto channelUtf8 = channel.toUtf8(); + quint16 channelLength = channelUtf8.length(); + packetList->writePrimitive(channelLength); + packetList->write(channelUtf8); + + auto messageUtf8 = message.toUtf8(); + quint16 messageLength = messageUtf8.length(); + packetList->writePrimitive(messageLength); + packetList->write(messageUtf8); + + nodeList->sendPacketList(std::move(packetList), *node); + }); +} + +void MessagesMixer::handleMessagesSubscribe(QSharedPointer packetList, SharedNodePointer senderNode) { + Q_ASSERT(packetList->getType() == PacketType::MessagesSubscribe); + QString channel = QString::fromUtf8(packetList->getMessage()); + qDebug() << "Node [" << senderNode->getUUID() << "] subscribed to channel:" << channel; + _channelSubscribers[channel] << senderNode->getUUID(); +} + +void MessagesMixer::handleMessagesUnsubscribe(QSharedPointer packetList, SharedNodePointer senderNode) { + Q_ASSERT(packetList->getType() == PacketType::MessagesUnsubscribe); + QString channel = QString::fromUtf8(packetList->getMessage()); + qDebug() << "Node [" << senderNode->getUUID() << "] unsubscribed from channel:" << channel; + + if (_channelSubscribers.contains(channel)) { + _channelSubscribers[channel].remove(senderNode->getUUID()); + } +} + +// FIXME - make these stats relevant +void MessagesMixer::sendStatsPacket() { + QJsonObject statsObject; + QJsonObject messagesObject; + auto nodeList = DependencyManager::get(); + // add stats for each listerner + nodeList->eachNode([&](const SharedNodePointer& node) { + QJsonObject messagesStats; + + // add the key to ask the domain-server for a username replacement, if it has it + messagesStats[USERNAME_UUID_REPLACEMENT_STATS_KEY] = uuidStringWithoutCurlyBraces(node->getUUID()); + messagesStats["outbound_kbps"] = node->getOutboundBandwidth(); + messagesStats["inbound_kbps"] = node->getInboundBandwidth(); + + messagesObject[uuidStringWithoutCurlyBraces(node->getUUID())] = messagesStats; + }); + + statsObject["messages"] = messagesObject; + ThreadedAssignment::addPacketStatsAndSendStatsPacket(statsObject); +} + +void MessagesMixer::run() { + ThreadedAssignment::commonInit(MESSAGES_MIXER_LOGGING_NAME, NodeType::MessagesMixer); + + auto nodeList = DependencyManager::get(); + nodeList->addNodeTypeToInterestSet(NodeType::Agent); + + // The messages-mixer currently does currently have any domain settings. If it did, they would be + // synchronously grabbed here. +} diff --git a/assignment-client/src/messages/MessagesMixer.h b/assignment-client/src/messages/MessagesMixer.h new file mode 100644 index 0000000000..65419a8ca6 --- /dev/null +++ b/assignment-client/src/messages/MessagesMixer.h @@ -0,0 +1,41 @@ +// +// MessagesMixer.h +// assignment-client/src/messages +// +// Created by Brad hefta-Gaub on 11/16/2015. +// Copyright 2015 High Fidelity, Inc. +// +// The avatar mixer receives head, hand and positional data from all connected +// nodes, and broadcasts that data back to them, every BROADCAST_INTERVAL ms. +// +// Distributed under the Apache License, Version 2.0. +// See the accompanying file LICENSE or http://www.apache.org/licenses/LICENSE-2.0.html +// + +#ifndef hifi_MessagesMixer_h +#define hifi_MessagesMixer_h + +#include + +/// Handles assignments of type MessagesMixer - distribution of avatar data to various clients +class MessagesMixer : public ThreadedAssignment { + Q_OBJECT +public: + MessagesMixer(NLPacket& packet); + ~MessagesMixer(); + +public slots: + void run(); + void nodeKilled(SharedNodePointer killedNode); + void sendStatsPacket(); + +private slots: + void handleMessages(QSharedPointer packetList, SharedNodePointer senderNode); + void handleMessagesSubscribe(QSharedPointer packetList, SharedNodePointer senderNode); + void handleMessagesUnsubscribe(QSharedPointer packetList, SharedNodePointer senderNode); + +private: + QHash> _channelSubscribers; +}; + +#endif // hifi_MessagesMixer_h diff --git a/assignment-client/src/octree/OctreeServer.cpp b/assignment-client/src/octree/OctreeServer.cpp index ad3df11474..84749bd975 100644 --- a/assignment-client/src/octree/OctreeServer.cpp +++ b/assignment-client/src/octree/OctreeServer.cpp @@ -953,7 +953,6 @@ bool OctreeServer::readConfiguration() { if (domainHandler.getSettingsObject().isEmpty()) { qDebug() << "Failed to retreive settings object from domain-server. Bailing on assignment."; - setFinished(true); return false; } @@ -1086,12 +1085,16 @@ void OctreeServer::run() { auto nodeList = DependencyManager::get(); nodeList->setOwnerType(getMyNodeType()); - // use common init to setup common timers and logging commonInit(getMyLoggingServerTargetName(), getMyNodeType()); + + // we need to ask the DS about agents so we can ping/reply with them + nodeList->addNodeTypeToInterestSet(NodeType::Agent); // read the configuration from either the payload or the domain server configuration if (!readConfiguration()) { + qDebug() << "OctreeServer bailing on run since readConfiguration has failed."; + setFinished(true); return; // bailing on run, because readConfiguration failed } @@ -1100,10 +1103,6 @@ void OctreeServer::run() { connect(nodeList.data(), SIGNAL(nodeAdded(SharedNodePointer)), SLOT(nodeAdded(SharedNodePointer))); connect(nodeList.data(), SIGNAL(nodeKilled(SharedNodePointer)), SLOT(nodeKilled(SharedNodePointer))); - - // we need to ask the DS about agents so we can ping/reply with them - nodeList->addNodeTypeToInterestSet(NodeType::Agent); - #ifndef WIN32 setvbuf(stdout, NULL, _IOLBF, 0); #endif diff --git a/cmake/externals/openvr/CMakeLists.txt b/cmake/externals/openvr/CMakeLists.txt index dea59f41a0..f9d0ef5a71 100644 --- a/cmake/externals/openvr/CMakeLists.txt +++ b/cmake/externals/openvr/CMakeLists.txt @@ -25,9 +25,14 @@ set(${EXTERNAL_NAME_UPPER}_INCLUDE_DIRS ${SOURCE_DIR}/headers CACHE TYPE INTERNA if (WIN32) - # FIXME need to account for different architectures - set(${EXTERNAL_NAME_UPPER}_LIBRARIES ${SOURCE_DIR}/lib/win32/openvr_api.lib CACHE TYPE INTERNAL) - add_paths_to_fixup_libs(${SOURCE_DIR}/bin/win32) + # FIXME need to account for different architectures + if ("${CMAKE_SIZEOF_VOID_P}" EQUAL "8") + set(${EXTERNAL_NAME_UPPER}_LIBRARIES ${SOURCE_DIR}/lib/win64/openvr_api.lib CACHE TYPE INTERNAL) + add_paths_to_fixup_libs(${SOURCE_DIR}/bin/win64) + else() + set(${EXTERNAL_NAME_UPPER}_LIBRARIES ${SOURCE_DIR}/lib/win32/openvr_api.lib CACHE TYPE INTERNAL) + add_paths_to_fixup_libs(${SOURCE_DIR}/bin/win32) + endif() elseif(APPLE) diff --git a/cmake/externals/sdl2/CMakeLists.txt b/cmake/externals/sdl2/CMakeLists.txt index decd1c6906..ee1d57d2e9 100644 --- a/cmake/externals/sdl2/CMakeLists.txt +++ b/cmake/externals/sdl2/CMakeLists.txt @@ -66,8 +66,15 @@ if (APPLE) elseif (WIN32) ExternalProject_Get_Property(${EXTERNAL_NAME} SOURCE_DIR) set(${EXTERNAL_NAME_UPPER}_INCLUDE_DIR ${SOURCE_DIR}/include CACHE PATH "Location of SDL2 include directory") - set(${EXTERNAL_NAME_UPPER}_LIBRARY_TEMP ${SOURCE_DIR}/lib/x86/SDL2.lib CACHE FILEPATH "Path to SDL2 library") - set(${EXTERNAL_NAME_UPPER}_DLL_PATH ${SOURCE_DIR}/lib/x86 CACHE PATH "Location of SDL2 DLL") + + if ("${CMAKE_SIZEOF_VOID_P}" EQUAL "8") + set(${EXTERNAL_NAME_UPPER}_LIBRARY_TEMP ${SOURCE_DIR}/lib/x64/SDL2.lib CACHE FILEPATH "Path to SDL2 library") + set(${EXTERNAL_NAME_UPPER}_DLL_PATH ${SOURCE_DIR}/lib/x64 CACHE PATH "Location of SDL2 DLL") + else() + set(${EXTERNAL_NAME_UPPER}_LIBRARY_TEMP ${SOURCE_DIR}/lib/x86/SDL2.lib CACHE FILEPATH "Path to SDL2 library") + set(${EXTERNAL_NAME_UPPER}_DLL_PATH ${SOURCE_DIR}/lib/x86 CACHE PATH "Location of SDL2 DLL") + endif() + else () ExternalProject_Get_Property(${EXTERNAL_NAME} INSTALL_DIR) set(${EXTERNAL_NAME_UPPER}_INCLUDE_DIR ${INSTALL_DIR}/include/SDL2 CACHE PATH "Location of SDL2 include directory") diff --git a/cmake/modules/FindOpenSSL.cmake b/cmake/modules/FindOpenSSL.cmake index db3b2ba477..2142322687 100644 --- a/cmake/modules/FindOpenSSL.cmake +++ b/cmake/modules/FindOpenSSL.cmake @@ -34,17 +34,26 @@ if (UNIX) endif () if (WIN32) - # http://www.slproweb.com/products/Win32OpenSSL.html - set(_OPENSSL_ROOT_HINTS ${OPENSSL_ROOT_DIR} $ENV{OPENSSL_ROOT_DIR} $ENV{HIFI_LIB_DIR}/openssl - "[HKEY_LOCAL_MACHINE\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Uninstall\\OpenSSL (32-bit)_is1;Inno Setup: App Path]" - "[HKEY_LOCAL_MACHINE\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Uninstall\\OpenSSL (64-bit)_is1;Inno Setup: App Path]" - ) + file(TO_CMAKE_PATH "$ENV{PROGRAMFILES}" _programfiles) - set(_OPENSSL_ROOT_PATHS "${_programfiles}/OpenSSL" "${_programfiles}/OpenSSL-Win32" "${_programfiles}/OpenSSL-Win64" - "C:/OpenSSL/" "C:/OpenSSL-Win32/" "C:/OpenSSL-Win64/" - ) + + if ("${CMAKE_SIZEOF_VOID_P}" EQUAL "8") + # http://www.slproweb.com/products/Win32OpenSSL.html + set(_OPENSSL_ROOT_HINTS ${OPENSSL_ROOT_DIR} $ENV{OPENSSL_ROOT_DIR} $ENV{HIFI_LIB_DIR}/openssl + "[HKEY_LOCAL_MACHINE\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Uninstall\\OpenSSL (64-bit)_is1;Inno Setup: App Path]" + ) + set(_OPENSSL_ROOT_PATHS "${_programfiles}/OpenSSL" "${_programfiles}/OpenSSL-Win64" "C:/OpenSSL/" "C:/OpenSSL-Win64/") + else() + # http://www.slproweb.com/products/Win32OpenSSL.html + set(_OPENSSL_ROOT_HINTS ${OPENSSL_ROOT_DIR} $ENV{OPENSSL_ROOT_DIR} $ENV{HIFI_LIB_DIR}/openssl + "[HKEY_LOCAL_MACHINE\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Uninstall\\OpenSSL (32-bit)_is1;Inno Setup: App Path]" + ) + set(_OPENSSL_ROOT_PATHS "${_programfiles}/OpenSSL" "${_programfiles}/OpenSSL-Win32" "C:/OpenSSL/" "C:/OpenSSL-Win32/") + endif() + unset(_programfiles) set(_OPENSSL_ROOT_HINTS_AND_PATHS HINTS ${_OPENSSL_ROOT_HINTS} PATHS ${_OPENSSL_ROOT_PATHS}) + else () include("${MACRO_DIR}/HifiLibrarySearchHints.cmake") hifi_library_search_hints("openssl") diff --git a/domain-server/src/DomainGatekeeper.cpp b/domain-server/src/DomainGatekeeper.cpp index d360ab4802..55f0fb2d2b 100644 --- a/domain-server/src/DomainGatekeeper.cpp +++ b/domain-server/src/DomainGatekeeper.cpp @@ -48,7 +48,8 @@ QUuid DomainGatekeeper::assignmentUUIDForPendingAssignment(const QUuid& tempUUID const NodeSet STATICALLY_ASSIGNED_NODES = NodeSet() << NodeType::AudioMixer << NodeType::AvatarMixer << NodeType::EntityServer - << NodeType::AssetServer; + << NodeType::AssetServer + << NodeType::MessagesMixer; void DomainGatekeeper::processConnectRequestPacket(QSharedPointer packet) { if (packet->getPayloadSize() == 0) { @@ -66,7 +67,7 @@ void DomainGatekeeper::processConnectRequestPacket(QSharedPointer pack } static const NodeSet VALID_NODE_TYPES { - NodeType::AudioMixer, NodeType::AvatarMixer, NodeType::AssetServer, NodeType::EntityServer, NodeType::Agent + NodeType::AudioMixer, NodeType::AvatarMixer, NodeType::AssetServer, NodeType::EntityServer, NodeType::Agent, NodeType::MessagesMixer }; if (!VALID_NODE_TYPES.contains(nodeConnection.nodeType)) { diff --git a/domain-server/src/DomainServer.cpp b/domain-server/src/DomainServer.cpp index b5fd9f2b20..d7bcec2431 100644 --- a/domain-server/src/DomainServer.cpp +++ b/domain-server/src/DomainServer.cpp @@ -272,6 +272,7 @@ void DomainServer::setupNodeListAndAssignments(const QUuid& sessionUUID) { packetReceiver.registerListener(PacketType::DomainListRequest, this, "processListRequestPacket"); packetReceiver.registerListener(PacketType::DomainServerPathQuery, this, "processPathQueryPacket"); packetReceiver.registerMessageListener(PacketType::NodeJsonStats, this, "processNodeJSONStatsPacket"); + packetReceiver.registerListener(PacketType::DomainDisconnectRequest, this, "processNodeDisconnectRequestPacket"); // NodeList won't be available to the settings manager when it is created, so call registerListener here packetReceiver.registerListener(PacketType::DomainSettingsRequest, &_settingsManager, "processSettingsRequestPacket"); @@ -553,7 +554,6 @@ void DomainServer::populateDefaultStaticAssignmentsExcludingTypes(const QSet(static_cast(defaultedType) + 1)) { if (!excludedTypes.contains(defaultedType) - && defaultedType != Assignment::UNUSED_0 && defaultedType != Assignment::UNUSED_1 && defaultedType != Assignment::AgentType) { @@ -1826,3 +1826,24 @@ void DomainServer::processPathQueryPacket(QSharedPointer packet) { } } } + +void DomainServer::processNodeDisconnectRequestPacket(QSharedPointer packet) { + // This packet has been matched to a source node and they're asking not to be in the domain anymore + auto limitedNodeList = DependencyManager::get(); + + const QUuid& nodeUUID = packet->getSourceID(); + + qDebug() << "Received a disconnect request from node with UUID" << nodeUUID; + + if (limitedNodeList->killNodeWithUUID(nodeUUID)) { + static auto removedNodePacket = NLPacket::create(PacketType::DomainServerRemovedNode, NUM_BYTES_RFC4122_UUID); + + removedNodePacket->reset(); + removedNodePacket->write(nodeUUID.toRfc4122()); + + // broadcast out the DomainServerRemovedNode message + limitedNodeList->eachNode([&limitedNodeList](const SharedNodePointer& otherNode){ + limitedNodeList->sendUnreliablePacket(*removedNodePacket, *otherNode); + }); + } +} diff --git a/domain-server/src/DomainServer.h b/domain-server/src/DomainServer.h index df42bf3ad9..e5b3d3b3fd 100644 --- a/domain-server/src/DomainServer.h +++ b/domain-server/src/DomainServer.h @@ -60,7 +60,8 @@ public slots: void processListRequestPacket(QSharedPointer packet, SharedNodePointer sendingNode); void processNodeJSONStatsPacket(QSharedPointer packetList, SharedNodePointer sendingNode); void processPathQueryPacket(QSharedPointer packet); - + void processNodeDisconnectRequestPacket(QSharedPointer packet); + private slots: void aboutToQuit(); diff --git a/domain-server/src/DomainServerSettingsManager.cpp b/domain-server/src/DomainServerSettingsManager.cpp index f650089486..88fc6a6cad 100644 --- a/domain-server/src/DomainServerSettingsManager.cpp +++ b/domain-server/src/DomainServerSettingsManager.cpp @@ -131,7 +131,7 @@ void DomainServerSettingsManager::setupConfigMap(const QStringList& argumentList appSettings.setValue(JSON_SETTINGS_VERSION_KEY, _descriptionVersion); } -QVariant DomainServerSettingsManager::valueOrDefaultValueForKeyPath(const QString &keyPath) { +QVariant DomainServerSettingsManager::valueOrDefaultValueForKeyPath(const QString& keyPath) { const QVariant* foundValue = valueForKeyPath(_configMap.getMergedConfig(), keyPath); if (foundValue) { diff --git a/examples/acScripts/ControlledAC.js b/examples/acScripts/ControlledAC.js index 41a8a2b257..cbc5f64b8f 100644 --- a/examples/acScripts/ControlledAC.js +++ b/examples/acScripts/ControlledAC.js @@ -38,11 +38,11 @@ var SHOW = 4; var HIDE = 5; var LOAD = 6; -Avatar.setPlayFromCurrentLocation(playFromCurrentLocation); -Avatar.setPlayerUseDisplayName(useDisplayName); -Avatar.setPlayerUseAttachments(useAttachments); -Avatar.setPlayerUseHeadModel(false); -Avatar.setPlayerUseSkeletonModel(useAvatarModel); +Recording.setPlayFromCurrentLocation(playFromCurrentLocation); +Recording.setPlayerUseDisplayName(useDisplayName); +Recording.setPlayerUseAttachments(useAttachments); +Recording.setPlayerUseHeadModel(false); +Recording.setPlayerUseSkeletonModel(useAvatarModel); function setupEntityViewer() { var entityViewerOffset = 10; @@ -96,25 +96,25 @@ function update(event) { if (!Agent.isAvatar) { Agent.isAvatar = true; } - if (!Avatar.isPlaying()) { - Avatar.startPlaying(); + if (!Recording.isPlaying()) { + Recording.startPlaying(); } - Avatar.setPlayerLoop(false); + Recording.setPlayerLoop(false); break; case PLAY_LOOP: print("Play loop"); if (!Agent.isAvatar) { Agent.isAvatar = true; } - if (!Avatar.isPlaying()) { - Avatar.startPlaying(); + if (!Recording.isPlaying()) { + Recording.startPlaying(); } - Avatar.setPlayerLoop(true); + Recording.setPlayerLoop(true); break; case STOP: print("Stop"); - if (Avatar.isPlaying()) { - Avatar.stopPlaying(); + if (Recording.isPlaying()) { + Recording.stopPlaying(); } break; case SHOW: @@ -125,15 +125,15 @@ function update(event) { break; case HIDE: print("Hide"); - if (Avatar.isPlaying()) { - Avatar.stopPlaying(); + if (Recording.isPlaying()) { + Recording.stopPlaying(); } Agent.isAvatar = false; break; case LOAD: print("Load"); if(clip_url !== null) { - Avatar.loadRecording(clip_url); + Recording.loadRecording(clip_url); } break; case DO_NOTHING: @@ -143,8 +143,8 @@ function update(event) { break; } - if (Avatar.isPlaying()) { - Avatar.play(); + if (Recording.isPlaying()) { + Recording.play(); } } diff --git a/examples/controllers/handControllerGrab.js b/examples/controllers/handControllerGrab.js index d308b3dc49..dd3a9a4b96 100644 --- a/examples/controllers/handControllerGrab.js +++ b/examples/controllers/handControllerGrab.js @@ -28,6 +28,8 @@ var TRIGGER_SMOOTH_RATIO = 0.1; // 0.0 disables smoothing of trigger value var TRIGGER_ON_VALUE = 0.4; var TRIGGER_OFF_VALUE = 0.15; +var BUMPER_ON_VALUE = 0.5; + // // distant manipulation // @@ -45,7 +47,7 @@ var PICK_MAX_DISTANCE = 500; // max length of pick-ray // near grabbing // -var GRAB_RADIUS = 0.3; // if the ray misses but an object is this close, it will still be selected +var GRAB_RADIUS = 0.03; // if the ray misses but an object is this close, it will still be selected var NEAR_GRABBING_ACTION_TIMEFRAME = 0.05; // how quickly objects move to their new position var NEAR_GRABBING_VELOCITY_SMOOTH_RATIO = 1.0; // adjust time-averaging of held object's velocity. 1.0 to disable. var NEAR_PICK_MAX_DISTANCE = 0.3; // max length of pick-ray for close grabbing to be selected @@ -53,6 +55,13 @@ var RELEASE_VELOCITY_MULTIPLIER = 1.5; // affects throwing things var PICK_BACKOFF_DISTANCE = 0.2; // helps when hand is intersecting the grabble object var NEAR_GRABBING_KINEMATIC = true; // force objects to be kinematic when near-grabbed +// +// equip +// + +var EQUIP_SPRING_SHUTOFF_DISTANCE = 0.05; +var EQUIP_SPRING_TIMEFRAME = 0.4; // how quickly objects move to their new position + // // other constants // @@ -68,7 +77,7 @@ var ZERO_VEC = { var NULL_ACTION_ID = "{00000000-0000-0000-000000000000}"; var MSEC_PER_SEC = 1000.0; -// these control how long an abandoned pointer line will hang around +// these control how long an abandoned pointer line or action will hang around var LIFETIME = 10; var ACTION_TTL = 15; // seconds var ACTION_TTL_REFRESH = 5; @@ -106,6 +115,12 @@ var STATE_CONTINUE_NEAR_TRIGGER = 7; var STATE_FAR_TRIGGER = 8; var STATE_CONTINUE_FAR_TRIGGER = 9; var STATE_RELEASE = 10; +var STATE_EQUIP_SEARCHING = 11; +var STATE_EQUIP = 12 +var STATE_CONTINUE_EQUIP_BD = 13; // equip while bumper is still held down +var STATE_CONTINUE_EQUIP = 14; +var STATE_WAITING_FOR_BUMPER_RELEASE = 15; +var STATE_EQUIP_SPRING = 16; function stateToName(state) { @@ -132,6 +147,18 @@ function stateToName(state) { return "continue_far_trigger"; case STATE_RELEASE: return "release"; + case STATE_EQUIP_SEARCHING: + return "equip_searching"; + case STATE_EQUIP: + return "equip"; + case STATE_CONTINUE_EQUIP_BD: + return "continue_equip_bd"; + case STATE_CONTINUE_EQUIP: + return "continue_equip"; + case STATE_WAITING_FOR_BUMPER_RELEASE: + return "waiting_for_bumper_release"; + case STATE_EQUIP_SPRING: + return "state_equip_spring"; } return "unknown"; @@ -182,6 +209,7 @@ function MyController(hand) { this.pointer = null; // entity-id of line object this.triggerValue = 0; // rolling average of trigger value this.rawTriggerValue = 0; + this.rawBumperValue = 0; this.offsetPosition = { x: 0.0, y: 0.0, z: 0.0 }; this.offsetRotation = { x: 0.0, y: 0.0, z: 0.0, w: 1.0 }; @@ -200,6 +228,9 @@ function MyController(hand) { case STATE_SEARCHING: this.search(); break; + case STATE_EQUIP_SEARCHING: + this.search(); + break; case STATE_DISTANCE_HOLDING: this.distanceHolding(); break; @@ -207,9 +238,18 @@ function MyController(hand) { this.continueDistanceHolding(); break; case STATE_NEAR_GRABBING: + case STATE_EQUIP: this.nearGrabbing(); break; + case STATE_WAITING_FOR_BUMPER_RELEASE: + this.waitingForBumperRelease(); + break; + case STATE_EQUIP_SPRING: + this.pullTowardEquipPosition() + break; case STATE_CONTINUE_NEAR_GRABBING: + case STATE_CONTINUE_EQUIP_BD: + case STATE_CONTINUE_EQUIP: this.continueNearGrabbing(); break; case STATE_NEAR_TRIGGER: @@ -281,10 +321,15 @@ function MyController(hand) { this.pointer = null; }; - this.eitherTrigger = function (value) { + this.triggerPress = function (value) { _this.rawTriggerValue = value; }; + this.bumperPress = function (value) { + _this.rawBumperValue = value; + }; + + this.updateSmoothedTrigger = function () { var triggerValue = this.rawTriggerValue; // smooth out trigger value @@ -305,23 +350,37 @@ function MyController(hand) { return triggerValue > TRIGGER_ON_VALUE; }; + this.bumperSqueezed = function() { + return _this.rawBumperValue > BUMPER_ON_VALUE; + } + + this.bumperReleased = function() { + return _this.rawBumperValue < BUMPER_ON_VALUE; + } + + this.off = function() { if (this.triggerSmoothedSqueezed()) { this.lastPickTime = 0; this.setState(STATE_SEARCHING); return; } + if (this.bumperSqueezed()) { + this.lastPickTime = 0; + this.setState(STATE_EQUIP_SEARCHING); + return; + } } this.search = function() { this.grabbedEntity = null; - //if this hand is the one that's disabled, we don't want to search for anything at all + // if this hand is the one that's disabled, we don't want to search for anything at all if (this.hand === disabledHand) { return; } - if (this.triggerSmoothedReleased()) { + if (this.state == STATE_SEARCHING ? this.triggerSmoothedReleased() : this.bumperReleased()) { this.setState(STATE_RELEASE); return; } @@ -334,8 +393,6 @@ function MyController(hand) { length: PICK_MAX_DISTANCE }; - this.lineOn(distantPickRay.origin, Vec3.multiply(distantPickRay.direction, LINE_LENGTH), NO_INTERSECT_COLOR); - // don't pick 60x per second. var pickRays = []; var now = Date.now(); @@ -398,7 +455,15 @@ function MyController(hand) { return; } else if (!intersection.properties.locked) { this.grabbedEntity = intersection.entityID; - this.setState(STATE_NEAR_GRABBING); + if (this.state == STATE_SEARCHING) { + this.setState(STATE_NEAR_GRABBING); + } else { // equipping + if (typeof grabbableData.spatialKey !== 'undefined') { + this.setState(STATE_EQUIP_SPRING); + } else { + this.setState(STATE_EQUIP); + } + } return; } } else if (! entityIsGrabbedByOther(intersection.entityID)) { @@ -407,8 +472,14 @@ function MyController(hand) { && !intersection.properties.locked) { // the hand is far from the intersected object. go into distance-holding mode this.grabbedEntity = intersection.entityID; - this.setState(STATE_DISTANCE_HOLDING); - return; + if (typeof grabbableData.spatialKey !== 'undefined' && this.state == STATE_EQUIP_SEARCHING) { + // if a distance pick in equip mode hits something with a spatialKey, equip it + this.setState(STATE_EQUIP_SPRING); + return; + } else if (this.state == STATE_SEARCHING) { + this.setState(STATE_DISTANCE_HOLDING); + return; + } } else if (grabbableData.wantsTrigger) { this.grabbedEntity = intersection.entityID; this.setState(STATE_FAR_TRIGGER); @@ -434,6 +505,7 @@ function MyController(hand) { var nearbyEntities = Entities.findEntities(handPosition, GRAB_RADIUS); var minDistance = PICK_MAX_DISTANCE; var i, props, distance, grabbableData; + this.grabbedEntity = null; for (i = 0; i < nearbyEntities.length; i++) { var grabbableDataForCandidate = getEntityCustomData(GRABBABLE_DATA_KEY, nearbyEntities[i], DEFAULT_GRABBABLE_DATA); @@ -483,16 +555,17 @@ function MyController(hand) { grabbableData = grabbableDataForCandidate; } } - if (this.grabbedEntity === null) { - return; - } - if (grabbableData.wantsTrigger) { - this.setState(STATE_NEAR_TRIGGER); - return; - } else if (!props.locked && props.collisionsWillMove) { - this.setState(STATE_NEAR_GRABBING); - return; + if (this.grabbedEntity !== null) { + if (grabbableData.wantsTrigger) { + this.setState(STATE_NEAR_TRIGGER); + return; + } else if (!props.locked && props.collisionsWillMove) { + this.setState(this.state == STATE_SEARCHING ? STATE_NEAR_GRABBING : STATE_EQUIP) + return; + } } + + this.lineOn(distantPickRay.origin, Vec3.multiply(distantPickRay.direction, LINE_LENGTH), NO_INTERSECT_COLOR); }; this.distanceHolding = function() { @@ -551,6 +624,16 @@ function MyController(hand) { var controllerHandInput = (this.hand === RIGHT_HAND) ? Controller.Standard.RightHand : Controller.Standard.LeftHand; var handRotation = Quat.multiply(MyAvatar.orientation, Controller.getPoseValue(controllerHandInput).rotation); var grabbedProperties = Entities.getEntityProperties(this.grabbedEntity, GRABBABLE_PROPERTIES); + var grabbableData = getEntityCustomData(GRABBABLE_DATA_KEY, this.grabbedEntity, DEFAULT_GRABBABLE_DATA); + + if (this.state == STATE_CONTINUE_DISTANCE_HOLDING && this.bumperSqueezed() && + typeof grabbableData.spatialKey !== 'undefined') { + var saveGrabbedID = this.grabbedEntity; + this.release(); + this.setState(STATE_EQUIP); + this.grabbedEntity = saveGrabbedID; + return; + } this.lineOn(handPosition, Vec3.subtract(grabbedProperties.position, handPosition), INTERSECT_COLOR); @@ -634,13 +717,12 @@ function MyController(hand) { return; } - if (this.triggerSmoothedReleased()) { + if (this.state == STATE_NEAR_GRABBING && this.triggerSmoothedReleased()) { this.setState(STATE_RELEASE); Entities.callEntityMethod(this.grabbedEntity, "releaseGrab"); return; } - this.lineOff(); var grabbedProperties = Entities.getEntityProperties(this.grabbedEntity, GRABBABLE_PROPERTIES); @@ -656,7 +738,8 @@ function MyController(hand) { var grabbableData = getEntityCustomData(GRABBABLE_DATA_KEY, this.grabbedEntity, DEFAULT_GRABBABLE_DATA); - if (grabbableData.spatialKey) { + if (this.state != STATE_NEAR_GRABBING && grabbableData.spatialKey) { + // if an object is "equipped" and has a spatialKey, use it. if (grabbableData.spatialKey.relativePosition) { this.offsetPosition = grabbableData.spatialKey.relativePosition; } @@ -686,7 +769,13 @@ function MyController(hand) { this.actionID = null; } else { this.actionTimeout = now + (ACTION_TTL * MSEC_PER_SEC); - this.setState(STATE_CONTINUE_NEAR_GRABBING); + if (this.state == STATE_NEAR_GRABBING) { + this.setState(STATE_CONTINUE_NEAR_GRABBING); + } else { + // equipping + this.setState(STATE_CONTINUE_EQUIP_BD); + } + if (this.hand === RIGHT_HAND) { Entities.callEntityMethod(this.grabbedEntity, "setRightHand"); } else { @@ -696,17 +785,30 @@ function MyController(hand) { } - this.currentHandControllerTipPosition = (this.hand === RIGHT_HAND) ? MyAvatar.rightHandTipPosition : MyAvatar.leftHandTipPosition;; + this.currentHandControllerTipPosition = + (this.hand === RIGHT_HAND) ? MyAvatar.rightHandTipPosition : MyAvatar.leftHandTipPosition; this.currentObjectTime = Date.now(); }; this.continueNearGrabbing = function() { - if (this.triggerSmoothedReleased()) { + if (this.state == STATE_CONTINUE_NEAR_GRABBING && this.triggerSmoothedReleased()) { this.setState(STATE_RELEASE); Entities.callEntityMethod(this.grabbedEntity, "releaseGrab"); return; } + if (this.state == STATE_CONTINUE_EQUIP_BD && this.bumperReleased()) { + this.setState(STATE_CONTINUE_EQUIP); + return; + } + if (this.state == STATE_CONTINUE_EQUIP && this.bumperSqueezed()) { + this.setState(STATE_WAITING_FOR_BUMPER_RELEASE); + return; + } + if (this.state == STATE_CONTINUE_NEAR_GRABBING && this.bumperSqueezed()) { + this.setState(STATE_CONTINUE_EQUIP_BD); + return; + } // Keep track of the fingertip velocity to impart when we release the object. // Note that the idea of using a constant 'tip' velocity regardless of the @@ -740,6 +842,66 @@ function MyController(hand) { } }; + this.waitingForBumperRelease = function() { + if (this.bumperReleased()) { + this.setState(STATE_RELEASE); + Entities.callEntityMethod(this.grabbedEntity, "releaseGrab"); + } + }; + + this.pullTowardEquipPosition = function() { + this.lineOff(); + + var grabbedProperties = Entities.getEntityProperties(this.grabbedEntity, GRABBABLE_PROPERTIES); + var grabbableData = getEntityCustomData(GRABBABLE_DATA_KEY, this.grabbedEntity, DEFAULT_GRABBABLE_DATA); + + // use a spring to pull the object to where it will be when equipped + var relativeRotation = { x: 0.0, y: 0.0, z: 0.0, w: 1.0 }; + var relativePosition = { x: 0.0, y: 0.0, z: 0.0 }; + if (grabbableData.spatialKey.relativePosition) { + relativePosition = grabbableData.spatialKey.relativePosition; + } + if (grabbableData.spatialKey.relativeRotation) { + relativeRotation = grabbableData.spatialKey.relativeRotation; + } + var handRotation = this.getHandRotation(); + var handPosition = this.getHandPosition(); + var targetRotation = Quat.multiply(handRotation, relativeRotation); + var offset = Vec3.multiplyQbyV(targetRotation, relativePosition); + var targetPosition = Vec3.sum(handPosition, offset); + + if (typeof this.equipSpringID === 'undefined' || + this.equipSpringID === null || + this.equipSpringID === NULL_ACTION_ID) { + this.equipSpringID = Entities.addAction("spring", this.grabbedEntity, { + targetPosition: targetPosition, + linearTimeScale: EQUIP_SPRING_TIMEFRAME, + targetRotation: targetRotation, + angularTimeScale: EQUIP_SPRING_TIMEFRAME, + ttl: ACTION_TTL + }); + if (this.equipSpringID === NULL_ACTION_ID) { + this.equipSpringID = null; + this.setState(STATE_OFF); + return; + } + } else { + Entities.updateAction(this.grabbedEntity, this.equipSpringID, { + targetPosition: targetPosition, + linearTimeScale: EQUIP_SPRING_TIMEFRAME, + targetRotation: targetRotation, + angularTimeScale: EQUIP_SPRING_TIMEFRAME, + ttl: ACTION_TTL + }); + } + + if (Vec3.distance(grabbedProperties.position, targetPosition) < EQUIP_SPRING_SHUTOFF_DISTANCE) { + Entities.deleteAction(this.grabbedEntity, this.equipSpringID); + this.equipSpringID = null; + this.setState(STATE_EQUIP); + } + }; + this.nearTrigger = function() { if (this.triggerSmoothedReleased()) { this.setState(STATE_RELEASE); @@ -919,6 +1081,7 @@ function MyController(hand) { } Entities.editEntity(entityID, whileHeldProperties); } + setEntityCustomData(GRAB_USER_DATA_KEY, entityID, data); return data; }; @@ -948,8 +1111,12 @@ var leftController = new MyController(LEFT_HAND); var MAPPING_NAME = "com.highfidelity.handControllerGrab"; var mapping = Controller.newMapping(MAPPING_NAME); -mapping.from([Controller.Standard.RB, Controller.Standard.RT]).peek().to(rightController.eitherTrigger); -mapping.from([Controller.Standard.LB, Controller.Standard.LT]).peek().to(leftController.eitherTrigger); +mapping.from([Controller.Standard.RT]).peek().to(rightController.triggerPress); +mapping.from([Controller.Standard.LT]).peek().to(leftController.triggerPress); + +mapping.from([Controller.Standard.RB]).peek().to(rightController.bumperPress); +mapping.from([Controller.Standard.LB]).peek().to(leftController.bumperPress); + Controller.enableMapping(MAPPING_NAME); diff --git a/examples/edit.js b/examples/edit.js index 7a16030afc..74551384c9 100644 --- a/examples/edit.js +++ b/examples/edit.js @@ -1617,6 +1617,11 @@ PropertiesTool = function(opts) { pushCommandForSelections(); selectionManager._update(); } + } else if (data.action == "previewCamera") { + if (selectionManager.hasSelection()) { + Camera.mode = "entity"; + Camera.cameraEntity = selectionManager.selections[0]; + } } else if (data.action == "rescaleDimensions") { var multiplier = data.percentage / 100; if (selectionManager.hasSelection()) { diff --git a/examples/entityScripts/createRecorder.js b/examples/entityScripts/createRecorder.js new file mode 100644 index 0000000000..7f89898ceb --- /dev/null +++ b/examples/entityScripts/createRecorder.js @@ -0,0 +1,21 @@ +var rotation = Quat.safeEulerAngles(Camera.getOrientation()); +rotation = Quat.fromPitchYawRollDegrees(0, rotation.y, 0); +var center = Vec3.sum(MyAvatar.position, Vec3.multiply(6, Quat.getFront(rotation))); + +var recordAreaEntity = Entities.addEntity({ + name: 'recorderEntity', + dimensions: { + x: 10, + y: 10, + z: 10 + }, + type: 'Box', + position: center, + color: { + red: 255, + green: 255, + blue: 255 + }, + visible: true, + script: "https://hifi-public.s3.amazonaws.com/sam/record/recordingEntityScript.js", +}); \ No newline at end of file diff --git a/examples/entityScripts/recordingEntityScript.js b/examples/entityScripts/recordingEntityScript.js new file mode 100644 index 0000000000..1b74466c4c --- /dev/null +++ b/examples/entityScripts/recordingEntityScript.js @@ -0,0 +1,106 @@ +// +// recordingEntityScript.js +// examples/entityScripts +// +// Created by Alessandro Signa on 11/12/15. +// Copyright 2015 High Fidelity, Inc. +// + +// All the avatars in the area when the master presses the button will start/stop recording. +// + +// Distributed under the Apache License, Version 2.0. +// See the accompanying file LICENSE or http://www.apache.org/licenses/LICENSE-2.0.html + + +(function () { + + HIFI_PUBLIC_BUCKET = "http://s3.amazonaws.com/hifi-public/"; + Script.include(HIFI_PUBLIC_BUCKET + "scripts/libraries/utils.js"); + + var insideRecorderArea = false; + var enteredInTime = false; + var isAvatarRecording = false; + var _this; + + function recordingEntity() { + _this = this; + return; + } + + function update() { + var isRecordingStarted = getEntityCustomData("recordingKey", _this.entityID, { isRecordingStarted: false }).isRecordingStarted; + if (isRecordingStarted && !isAvatarRecording) { + _this.startRecording(); + } else if ((!isRecordingStarted && isAvatarRecording) || (isAvatarRecording && !insideRecorderArea)) { + _this.stopRecording(); + } else if (!isRecordingStarted && insideRecorderArea && !enteredInTime) { + //if an avatar enters the zone while a recording is started he will be able to participate to the next group recording + enteredInTime = true; + } + }; + + recordingEntity.prototype = { + + preload: function (entityID) { + print("RECORDING ENTITY PRELOAD"); + this.entityID = entityID; + + var entityProperties = Entities.getEntityProperties(_this.entityID); + if (!entityProperties.ignoreForCollisions) { + Entities.editEntity(_this.entityID, { ignoreForCollisions: true }); + } + + //print(JSON.stringify(entityProperties)); + var recordingKey = getEntityCustomData("recordingKey", _this.entityID, undefined); + if (recordingKey === undefined) { + setEntityCustomData("recordingKey", _this.entityID, { isRecordingStarted: false }); + } + + Script.update.connect(update); + }, + enterEntity: function (entityID) { + print("entering in the recording area"); + insideRecorderArea = true; + var isRecordingStarted = getEntityCustomData("recordingKey", _this.entityID, { isRecordingStarted: false }).isRecordingStarted; + if (!isRecordingStarted) { + //i'm in the recording area in time (before the event starts) + enteredInTime = true; + } + }, + leaveEntity: function (entityID) { + print("leaving the recording area"); + insideRecorderArea = false; + enteredInTime = false; + }, + + startRecording: function (entityID) { + if (enteredInTime && !isAvatarRecording) { + print("RECORDING STARTED"); + Recording.startRecording(); + isAvatarRecording = true; + } + }, + + stopRecording: function (entityID) { + if (isAvatarRecording) { + print("RECORDING ENDED"); + Recording.stopRecording(); + Recording.loadLastRecording(); + isAvatarRecording = false; + recordingFile = Window.save("Save recording to file", "./groupRecording", "Recordings (*.hfr)"); + if (!(recordingFile === "null" || recordingFile === null || recordingFile === "")) { + Recording.saveRecording(recordingFile); + } + } + }, + unload: function (entityID) { + print("RECORDING ENTITY UNLOAD"); + Script.update.disconnect(update); + } + } + + + + return new recordingEntity(); +}); \ No newline at end of file diff --git a/examples/entityScripts/recordingMaster.js b/examples/entityScripts/recordingMaster.js new file mode 100644 index 0000000000..71a92a05f3 --- /dev/null +++ b/examples/entityScripts/recordingMaster.js @@ -0,0 +1,102 @@ +// +// recordingMaster.js +// examples/entityScripts +// +// Created by Alessandro Signa on 11/12/15. +// Copyright 2015 High Fidelity, Inc. +// +// Run this script to find the recorder (created by crateRecorder.js) and drive the start/end of the recording for anyone who is inside the box +// +// Distributed under the Apache License, Version 2.0. +// See the accompanying file LICENSE or http://www.apache.org/licenses/LICENSE-2.0.html + + +HIFI_PUBLIC_BUCKET = "http://s3.amazonaws.com/hifi-public/"; +Script.include(HIFI_PUBLIC_BUCKET + "scripts/libraries/toolBars.js"); +Script.include(HIFI_PUBLIC_BUCKET + "scripts/libraries/utils.js"); + + + +var rotation = Quat.safeEulerAngles(Camera.getOrientation()); +rotation = Quat.fromPitchYawRollDegrees(0, rotation.y, 0); +var center = Vec3.sum(MyAvatar.position, Vec3.multiply(1, Quat.getFront(rotation))); + +var TOOL_ICON_URL = HIFI_PUBLIC_BUCKET + "images/tools/"; +var ALPHA_ON = 1.0; +var ALPHA_OFF = 0.7; +var COLOR_TOOL_BAR = { red: 0, green: 0, blue: 0 }; + +var toolBar = null; +var recordIcon; + +var isRecordingEntityFound = false; + +var isRecording = false; + +var recordAreaEntity = null; +findRecorder(); + +function findRecorder() { + foundEntities = Entities.findEntities(MyAvatar.position, 50); + for (var i = 0; i < foundEntities.length; i++) { + var name = Entities.getEntityProperties(foundEntities[i], "name").name; + if (name === "recorderEntity") { + recordAreaEntity = foundEntities[i]; + isRecordingEntityFound = true; + print("Found recorder Entity!"); + return; + } + } +} + +setupToolBar(); + +function setupToolBar() { + if (toolBar != null) { + print("Multiple calls to setupToolBar()"); + return; + } + Tool.IMAGE_HEIGHT /= 2; + Tool.IMAGE_WIDTH /= 2; + + toolBar = new ToolBar(0, 100, ToolBar.HORIZONTAL); //put the button in the up-left corner + + toolBar.setBack(COLOR_TOOL_BAR, ALPHA_OFF); + + recordIcon = toolBar.addTool({ + imageURL: TOOL_ICON_URL + "recording-record.svg", + subImage: { x: 0, y: 0, width: Tool.IMAGE_WIDTH, height: Tool.IMAGE_HEIGHT }, + x: 0, y: 0, + width: Tool.IMAGE_WIDTH, + height: Tool.IMAGE_HEIGHT, + alpha: Recording.isPlaying() ? ALPHA_OFF : ALPHA_ON, + visible: isRecordingEntityFound, + }, true, isRecording); +} + +function mousePressEvent(event) { + clickedOverlay = Overlays.getOverlayAtPoint({ x: event.x, y: event.y }); + if (recordIcon === toolBar.clicked(clickedOverlay, false)) { + if (!isRecording) { + print("I'm the master. I want to start recording"); + isRecording = true; + setEntityCustomData("recordingKey", recordAreaEntity, {isRecordingStarted: true}); + + } else { + print("I want to stop recording"); + isRecording = false; + setEntityCustomData("recordingKey", recordAreaEntity, {isRecordingStarted: false}); + + } + } +} + + +function cleanup() { + toolBar.cleanup(); +} + + + + Script.scriptEnding.connect(cleanup); + Controller.mousePressEvent.connect(mousePressEvent); \ No newline at end of file diff --git a/examples/entityScripts/synchronizerEntityScript.js b/examples/entityScripts/synchronizerEntityScript.js new file mode 100644 index 0000000000..82dd954381 --- /dev/null +++ b/examples/entityScripts/synchronizerEntityScript.js @@ -0,0 +1,90 @@ +// +// synchronizerEntityScript.js +// examples/entityScripts +// +// Created by Alessandro Signa on 11/12/15. +// Copyright 2015 High Fidelity, Inc. +// + +// This script shows how to create a synchronized event between avatars trhough an entity. +// It works using the entity's userData: the master change its value and every client checks it every frame +// This entity prints a message when the event starts and when it ends. +// The client running synchronizerMaster.js is the event master and it decides when the event starts/ends by pressing a button. +// All the avatars in the area when the master presses the button will receive a message. +// + +// Distributed under the Apache License, Version 2.0. +// See the accompanying file LICENSE or http://www.apache.org/licenses/LICENSE-2.0.html + + + + +(function() { + var insideArea = false; + var isJoiningTheEvent = false; + var _this; + + + + function ParamsEntity() { + _this = this; + return; + } + + + ParamsEntity.prototype = { + update: function(){ + var userData = JSON.parse(Entities.getEntityProperties(_this.entityID, ["userData"]).userData); + var valueToCheck = userData.myKey.valueToCheck; + if(valueToCheck && !isJoiningTheEvent){ + _this.sendMessage(); + }else if((!valueToCheck && isJoiningTheEvent) || (isJoiningTheEvent && !insideArea)){ + _this.stopMessage(); + } + }, + preload: function(entityID) { + print('entity loaded') + this.entityID = entityID; + Script.update.connect(_this.update); + }, + enterEntity: function(entityID) { + print("enterEntity("+entityID+")"); + var userData = JSON.parse(Entities.getEntityProperties(_this.entityID, ["userData"]).userData); + var valueToCheck = userData.myKey.valueToCheck; + if(!valueToCheck){ + //i'm in the area in time (before the event starts) + insideArea = true; + } + change(entityID); + }, + leaveEntity: function(entityID) { + print("leaveEntity("+entityID+")"); + Entities.editEntity(entityID, { color: { red: 255, green: 190, blue: 20} }); + insideArea = false; + }, + + sendMessage: function(myID){ + if(insideArea && !isJoiningTheEvent){ + print("The event started"); + isJoiningTheEvent = true; + } + }, + + stopMessage: function(myID){ + if(isJoiningTheEvent){ + print("The event ended"); + isJoiningTheEvent = false; + } + }, + clean: function(entityID) { + Script.update.disconnect(_this.update); + } + } + + function change(entityID) { + Entities.editEntity(entityID, { color: { red: 255, green: 100, blue: 220} }); + } + + + return new ParamsEntity(); +}); diff --git a/examples/entityScripts/synchronizerMaster.js b/examples/entityScripts/synchronizerMaster.js new file mode 100644 index 0000000000..8b6c8c2b8b --- /dev/null +++ b/examples/entityScripts/synchronizerMaster.js @@ -0,0 +1,117 @@ +// +// synchronizerMaster.js +// examples/entityScripts +// +// Created by Alessandro Signa on 11/12/15. +// Copyright 2015 High Fidelity, Inc. +// +// Run this script to spawn a box (synchronizer) and drive the start/end of the event for anyone who is inside the box +// +// Distributed under the Apache License, Version 2.0. +// See the accompanying file LICENSE or http://www.apache.org/licenses/LICENSE-2.0.html + +var PARAMS_SCRIPT_URL = Script.resolvePath('synchronizerEntityScript.js'); + + +HIFI_PUBLIC_BUCKET = "http://s3.amazonaws.com/hifi-public/"; +Script.include("../libraries/toolBars.js"); +Script.include("../libraries/utils.js"); + + + +var rotation = Quat.safeEulerAngles(Camera.getOrientation()); +rotation = Quat.fromPitchYawRollDegrees(0, rotation.y, 0); +var center = Vec3.sum(MyAvatar.position, Vec3.multiply(1, Quat.getFront(rotation))); + +var TOOL_ICON_URL = HIFI_PUBLIC_BUCKET + "images/tools/"; +var ALPHA_ON = 1.0; +var ALPHA_OFF = 0.7; +var COLOR_TOOL_BAR = { red: 0, green: 0, blue: 0 }; + +var toolBar = null; +var recordIcon; + + + +var isHappening = false; + +var testEntity = Entities.addEntity({ + name: 'paramsTestEntity', + dimensions: { + x: 2, + y: 1, + z: 2 + }, + type: 'Box', + position: center, + color: { + red: 255, + green: 255, + blue: 255 + }, + visible: true, + ignoreForCollisions: true, + script: PARAMS_SCRIPT_URL, + + userData: JSON.stringify({ + myKey: { + valueToCheck: false + } + }) +}); + + +setupToolBar(); + +function setupToolBar() { + if (toolBar != null) { + print("Multiple calls to setupToolBar()"); + return; + } + Tool.IMAGE_HEIGHT /= 2; + Tool.IMAGE_WIDTH /= 2; + + toolBar = new ToolBar(0, 0, ToolBar.HORIZONTAL); //put the button in the up-left corner + + toolBar.setBack(COLOR_TOOL_BAR, ALPHA_OFF); + + recordIcon = toolBar.addTool({ + imageURL: TOOL_ICON_URL + "recording-record.svg", + subImage: { x: 0, y: 0, width: Tool.IMAGE_WIDTH, height: Tool.IMAGE_HEIGHT }, + x: 0, y: 0, + width: Tool.IMAGE_WIDTH, + height: Tool.IMAGE_HEIGHT, + alpha: MyAvatar.isPlaying() ? ALPHA_OFF : ALPHA_ON, + visible: true + }, true, isHappening); + +} + +function mousePressEvent(event) { + clickedOverlay = Overlays.getOverlayAtPoint({ x: event.x, y: event.y }); + if (recordIcon === toolBar.clicked(clickedOverlay, false)) { + if (!isHappening) { + print("I'm the event master. I want the event starts"); + isHappening = true; + setEntityCustomData("myKey", testEntity, {valueToCheck: true}); + + } else { + print("I want the event stops"); + isHappening = false; + setEntityCustomData("myKey", testEntity, {valueToCheck: false}); + + } + } +} + + +function cleanup() { + toolBar.cleanup(); + Entities.callEntityMethod(testEntity, 'clean'); //have to call this before deleting to avoid the JSON warnings + Entities.deleteEntity(testEntity); +} + + + + Script.scriptEnding.connect(cleanup); + Controller.mousePressEvent.connect(mousePressEvent); \ No newline at end of file diff --git a/examples/example/messagesExample.js b/examples/example/messagesExample.js new file mode 100644 index 0000000000..390549a135 --- /dev/null +++ b/examples/example/messagesExample.js @@ -0,0 +1,39 @@ +var totalTime = 0; +var unsubscribedForTime = 0; +var subscribedForTime = 0; +var subscribed = false; +var SWITCH_SUBSCRIPTION_TIME = 10; +Script.update.connect(function (deltaTime) { + var channel = "example"; + totalTime += deltaTime; + if (!subscribed) { + unsubscribedForTime += deltaTime; + } else { + subscribedForTime += deltaTime; + } + + // if we've been unsubscribed for SWITCH_SUBSCRIPTION_TIME seconds, subscribe + if (!subscribed && unsubscribedForTime > SWITCH_SUBSCRIPTION_TIME) { + print("---- subscribing ----"); + Messages.subscribe(channel); + subscribed = true; + subscribedForTime = 0; + } + + // if we've been subscribed for SWITCH_SUBSCRIPTION_TIME seconds, unsubscribe + if (subscribed && subscribedForTime > SWITCH_SUBSCRIPTION_TIME) { + print("---- unsubscribing ----"); + Messages.unsubscribe(channel); + subscribed = false; + unsubscribedForTime = 0; + } + + // Even if not subscribed, still publish + var message = "update() deltaTime:" + deltaTime; + Messages.sendMessage(channel, message); +}); + + +Messages.messageReceived.connect(function (channel, message, senderID) { + print("message received on channel:" + channel + ", message:" + message + ", senderID:" + senderID); +}); \ No newline at end of file diff --git a/examples/example/messagesReceiverExample.js b/examples/example/messagesReceiverExample.js new file mode 100644 index 0000000000..b9f9e54bca --- /dev/null +++ b/examples/example/messagesReceiverExample.js @@ -0,0 +1,7 @@ +print("---- subscribing ----"); +Messages.subscribe("example"); + +Messages.messageReceived.connect(function (channel, message, senderID) { + print("message received on channel:" + channel + ", message:" + message + ", senderID:" + senderID); +}); + diff --git a/examples/example/securityCamera.js b/examples/example/securityCamera.js new file mode 100644 index 0000000000..6f5ca549cd --- /dev/null +++ b/examples/example/securityCamera.js @@ -0,0 +1,57 @@ +// +// securityCamera.js +// examples/example +// +// Created by Thijs Wenker on November 4, 2015 +// Copyright 2015 High Fidelity, Inc. +// +// Distributed under the Apache License, Version 2.0. +// See the accompanying file LICENSE or http://www.apache.org/licenses/LICENSE-2.0.html +// + +const CAMERA_OFFSET = {x: 0, y: 4, z: -14}; +const LOOKAT_START_OFFSET = {x: -10, y: 0, z: 14}; +const LOOKAT_END_OFFSET = {x: 10, y: 0, z: 14}; +const TINY_VALUE = 0.001; + +var lookatTargets = [Vec3.sum(MyAvatar.position, LOOKAT_START_OFFSET), Vec3.sum(MyAvatar.position, LOOKAT_END_OFFSET)]; +var currentTarget = 0; + +var forward = true; + +var oldCameraMode = Camera.mode; + +var cameraLookAt = function(cameraPos, lookAtPos) { + var lookAtRaw = Quat.lookAt(cameraPos, lookAtPos, Vec3.UP); + lookAtRaw.w = -lookAtRaw.w; + return lookAtRaw; +}; + +cameraEntity = Entities.addEntity({ + type: "Box", + visible: false, + position: Vec3.sum(MyAvatar.position, CAMERA_OFFSET) +}); + +Camera.mode = "entity"; +Camera.cameraEntity = cameraEntity; + +Script.update.connect(function(deltaTime) { + var cameraProperties = Entities.getEntityProperties(cameraEntity, ["position", "rotation"]); + var targetOrientation = cameraLookAt(cameraProperties.position, lookatTargets[currentTarget]); + if (Math.abs(targetOrientation.x - cameraProperties.rotation.x) < TINY_VALUE && + Math.abs(targetOrientation.y - cameraProperties.rotation.y) < TINY_VALUE && + Math.abs(targetOrientation.z - cameraProperties.rotation.z) < TINY_VALUE && + Math.abs(targetOrientation.w - cameraProperties.rotation.w) < TINY_VALUE) { + + currentTarget = (currentTarget + 1) % lookatTargets.length; + return; + } + Entities.editEntity(cameraEntity, {rotation: Quat.mix(cameraProperties.rotation, targetOrientation, deltaTime / 3)}); +}); + +Script.scriptEnding.connect(function() { + Entities.deleteEntity(cameraEntity); + Camera.mode = oldCameraMode; + Camera.cameraEntity = null; +}); diff --git a/examples/growth.js b/examples/growth.js new file mode 100644 index 0000000000..8bd49d92c5 --- /dev/null +++ b/examples/growth.js @@ -0,0 +1,122 @@ +// +// growth.js +// +// Created by Zachary Pomerantz 11/2/2015. +// Copyright 2015 High Fidelity, Inc. +// +// Distributed under the Apache License, Version 2.0. +// See the accompanying file LICENSE or http://www.apache.org/licenses/LICENSE-2.0.html +// +var RADIUS = 5; +var LIFETIME = 60; // only keep these entities around for a minute + +var SIZE = 1; // starting size +var SIZE_MAX = 3; +var SIZE_MIN = 0.5; + +var center = MyAvatar.position; +var positions = [ + Vec3.sum(center, Vec3.multiply(RADIUS, Vec3.FRONT)), + Vec3.sum(center, Vec3.multiply(-RADIUS, Vec3.FRONT)), + Vec3.sum(center, Vec3.multiply(RADIUS, Vec3.RIGHT)), + Vec3.sum(center, Vec3.multiply(-RADIUS, Vec3.RIGHT)), +]; + +// Create spheres around the avatar +var spheres = map(positions, getSphere); + +// Toggle sphere growth +each(spheres, function(sphere) { + var state = { + size: SIZE, + grow: true, // grow/shrink + state: false, // mutate/pause + }; + + // Grow or shrink on click + Script.addEventHandler(sphere, 'clickReleaseOnEntity', toggleGrowth); + // TODO: Toggle on collisions as well + + function toggleGrowth() { + if (state.state) { + stop(); + } else { + start(); + } + } + + function start() { + var verb = state.grow ? 'grow' : 'shrink'; + print('starting to '+verb+' '+sphere); + + Script.update.connect(grow); + state.state = true; + } + + function stop() { + print('stopping '+sphere); + + Script.update.disconnect(grow); + state.state = false; + state.grow = !state.grow; + } + + function grow(delta) { + if ((state.grow && state.size > SIZE_MAX) || // cannot grow more + (!state.grow && state.size < SIZE_MIN)) { // cannot shrink more + stop(); + return; + } + + state.size += (state.grow ? 1 : -1) * delta; // grow/shrink + Entities.editEntity(sphere, { dimensions: getDimensions(state.size) }); + } +}); + +Script.scriptEnding.connect(function() { // cleanup + each(spheres, function(sphere) { + Entities.deleteEntity(sphere); + }); +}); + +// NOTE: Everything below this line is a helper. +// The basic logic of this example is entirely above this note. + +// Entity helpers + +function getSphere(position) { + return Entities.addEntity({ + type: 'Sphere', + position: position, + dimensions: getDimensions(SIZE), + color: getColor(), + gravity: Vec3.ZERO, + lifetime: LIFETIME, + ignoreCollisions: true, + }); +} + +function getDimensions(size) { + return { x: size, y: size, z: size }; +} + +function getColor() { + return { red: getValue(), green: getValue(), blue: getValue() }; + function getValue() { return Math.floor(Math.random() * 255); } +} + +// Functional helpers + +function map(list, iterator) { + var result = []; + for (var i = 0; i < list.length; i++) { + result.push(iterator(list[i], i, list)); + } + return result; +} + +function each(list, iterator) { + for (var i = 0; i < list.length; i++) { + iterator(list[i], i, list); + } +} diff --git a/examples/html/entityProperties.html b/examples/html/entityProperties.html index 412b413b2b..bc6a6920d1 100644 --- a/examples/html/entityProperties.html +++ b/examples/html/entityProperties.html @@ -382,7 +382,7 @@ var elHyperlinkHref = document.getElementById("property-hyperlink-href"); var elHyperlinkDescription = document.getElementById("property-hyperlink-description"); - + var elPreviewCameraButton = document.getElementById("preview-camera-button"); if (window.EventBridge !== undefined) { EventBridge.scriptEventReceived.connect(function(data) { @@ -931,6 +931,12 @@ action: "centerAtmosphereToZone", })); }); + elPreviewCameraButton.addEventListener("click", function() { + EventBridge.emitWebEvent(JSON.stringify({ + type: "action", + action: "previewCamera" + })); + }); window.onblur = function() { // Fake a change event @@ -1032,7 +1038,7 @@
- +
@@ -1044,6 +1050,7 @@
+
diff --git a/examples/libraries/entityPropertyDialogBox.js b/examples/libraries/entityPropertyDialogBox.js index 4fd24756e0..39ffbed629 100644 --- a/examples/libraries/entityPropertyDialogBox.js +++ b/examples/libraries/entityPropertyDialogBox.js @@ -204,7 +204,7 @@ EntityPropertyDialogBox = (function () { array.push({ label: "Collisions Will Move:", type: "checkbox", value: properties.collisionsWillMove }); index++; array.push({ label: "Collision Sound URL:", value: properties.collisionSoundURL }); - index++; + index++; array.push({ label: "Lifetime:", value: properties.lifetime.toFixed(decimals) }); index++; @@ -260,6 +260,10 @@ EntityPropertyDialogBox = (function () { array.push({ label: "Cutoff (in degrees):", value: properties.cutoff }); index++; } + + array.push({ label: "", type: "inlineButton", buttonLabel: "Preview Camera", name: "previewCamera" }); + index++; + array.push({ button: "Cancel" }); index++; @@ -268,6 +272,11 @@ EntityPropertyDialogBox = (function () { }; Window.inlineButtonClicked.connect(function (name) { + if (name == "previewCamera") { + Camera.mode = "entity"; + Camera.cameraEntity = propertiesForEditedEntity.id; + } + if (name == "resetDimensions") { Window.reloadNonBlockingForm([ { value: propertiesForEditedEntity.naturalDimensions.x.toFixed(decimals), oldIndex: dimensionX }, diff --git a/examples/libraries/utils.js b/examples/libraries/utils.js index 25900471c1..5d14bfb7dd 100644 --- a/examples/libraries/utils.js +++ b/examples/libraries/utils.js @@ -11,6 +11,12 @@ vec3toStr = function(v, digits) { return "{ " + v.x.toFixed(digits) + ", " + v.y.toFixed(digits) + ", " + v.z.toFixed(digits)+ " }"; } +quatToStr = function(q, digits) { + if (!digits) { digits = 3; } + return "{ " + q.w.toFixed(digits) + ", " + q.x.toFixed(digits) + ", " + + q.y.toFixed(digits) + ", " + q.z.toFixed(digits)+ " }"; +} + vec3equal = function(v0, v1) { return (v0.x == v1.x) && (v0.y == v1.y) && (v0.z == v1.z); } @@ -51,7 +57,7 @@ addLine = function(origin, vector, color) { // FIXME fetch from a subkey of user data to support non-destructive modifications setEntityUserData = function(id, data) { var json = JSON.stringify(data) - Entities.editEntity(id, { userData: json }); + Entities.editEntity(id, { userData: json }); } // FIXME do non-destructive modification of the existing user data @@ -60,7 +66,7 @@ getEntityUserData = function(id) { var properties = Entities.getEntityProperties(id, "userData"); if (properties.userData) { try { - results = JSON.parse(properties.userData); + results = JSON.parse(properties.userData); } catch(err) { logDebug(err); logDebug(properties.userData); diff --git a/examples/toybox/bubblewand/wand.js b/examples/toybox/bubblewand/wand.js index 4bdc789612..8036d9ead6 100644 --- a/examples/toybox/bubblewand/wand.js +++ b/examples/toybox/bubblewand/wand.js @@ -16,7 +16,7 @@ Script.include("../../libraries/utils.js"); - var BUBBLE_MODEL = "http://hifi-public.s3.amazonaws.com/models/bubblewand/bubble.fbx"; + var BUBBLE_MODEL = "http://hifi-content.s3.amazonaws.com/james/bubblewand/bubble.fbx"; var BUBBLE_INITIAL_DIMENSIONS = { x: 0.01, diff --git a/examples/utilities/record/recorder.js b/examples/utilities/record/recorder.js index 40476626e8..d08cdd68f3 100644 --- a/examples/utilities/record/recorder.js +++ b/examples/utilities/record/recorder.js @@ -15,11 +15,11 @@ Script.include("../../libraries/toolBars.js"); var recordingFile = "recording.rec"; function setPlayerOptions() { - MyAvatar.setPlayFromCurrentLocation(true); - MyAvatar.setPlayerUseDisplayName(false); - MyAvatar.setPlayerUseAttachments(false); - MyAvatar.setPlayerUseHeadModel(false); - MyAvatar.setPlayerUseSkeletonModel(false); + Recording.setPlayFromCurrentLocation(true); + Recording.setPlayerUseDisplayName(false); + Recording.setPlayerUseAttachments(false); + Recording.setPlayerUseHeadModel(false); + Recording.setPlayerUseSkeletonModel(false); } var windowDimensions = Controller.getViewportDimensions(); @@ -64,16 +64,16 @@ function setupToolBar() { x: 0, y: 0, width: Tool.IMAGE_WIDTH, height: Tool.IMAGE_HEIGHT, - alpha: MyAvatar.isPlaying() ? ALPHA_OFF : ALPHA_ON, + alpha: Recording.isPlaying() ? ALPHA_OFF : ALPHA_ON, visible: true - }, true, !MyAvatar.isRecording()); + }, true, !Recording.isRecording()); var playLoopWidthFactor = 1.65; playIcon = toolBar.addTool({ imageURL: TOOL_ICON_URL + "play-pause.svg", width: playLoopWidthFactor * Tool.IMAGE_WIDTH, height: Tool.IMAGE_HEIGHT, - alpha: (MyAvatar.isRecording() || MyAvatar.playerLength() === 0) ? ALPHA_OFF : ALPHA_ON, + alpha: (Recording.isRecording() || Recording.playerLength() === 0) ? ALPHA_OFF : ALPHA_ON, visible: true }, false); @@ -82,7 +82,7 @@ function setupToolBar() { subImage: { x: 0, y: 0, width: playLoopWidthFactor * Tool.IMAGE_WIDTH, height: Tool.IMAGE_HEIGHT }, width: playLoopWidthFactor * Tool.IMAGE_WIDTH, height: Tool.IMAGE_HEIGHT, - alpha: (MyAvatar.isRecording() || MyAvatar.playerLength() === 0) ? ALPHA_OFF : ALPHA_ON, + alpha: (Recording.isRecording() || Recording.playerLength() === 0) ? ALPHA_OFF : ALPHA_ON, visible: true }, false); @@ -93,7 +93,7 @@ function setupToolBar() { imageURL: TOOL_ICON_URL + "recording-save.svg", width: Tool.IMAGE_WIDTH, height: Tool.IMAGE_HEIGHT, - alpha: (MyAvatar.isRecording() || MyAvatar.isPlaying() || MyAvatar.playerLength() === 0) ? ALPHA_OFF : ALPHA_ON, + alpha: (Recording.isRecording() || Recording.isPlaying() || Recording.playerLength() === 0) ? ALPHA_OFF : ALPHA_ON, visible: true }, false); @@ -101,7 +101,7 @@ function setupToolBar() { imageURL: TOOL_ICON_URL + "recording-upload.svg", width: Tool.IMAGE_WIDTH, height: Tool.IMAGE_HEIGHT, - alpha: (MyAvatar.isRecording() || MyAvatar.isPlaying()) ? ALPHA_OFF : ALPHA_ON, + alpha: (Recording.isRecording() || Recording.isPlaying()) ? ALPHA_OFF : ALPHA_ON, visible: true }, false); } @@ -147,23 +147,23 @@ function setupTimer() { function updateTimer() { var text = ""; - if (MyAvatar.isRecording()) { - text = formatTime(MyAvatar.recorderElapsed()); + if (Recording.isRecording()) { + text = formatTime(Recording.recorderElapsed()); } else { - text = formatTime(MyAvatar.playerElapsed()) + " / " + - formatTime(MyAvatar.playerLength()); + text = formatTime(Recording.playerElapsed()) + " / " + + formatTime(Recording.playerLength()); } Overlays.editOverlay(timer, { text: text }) - toolBar.changeSpacing(text.length * 8 + ((MyAvatar.isRecording()) ? 15 : 0), spacing); + toolBar.changeSpacing(text.length * 8 + ((Recording.isRecording()) ? 15 : 0), spacing); - if (MyAvatar.isRecording()) { + if (Recording.isRecording()) { slider.pos = 1.0; - } else if (MyAvatar.playerLength() > 0) { - slider.pos = MyAvatar.playerElapsed() / MyAvatar.playerLength(); + } else if (Recording.playerLength() > 0) { + slider.pos = Recording.playerElapsed() / Recording.playerLength(); } Overlays.editOverlay(slider.foreground, { @@ -217,77 +217,77 @@ function moveUI() { function mousePressEvent(event) { clickedOverlay = Overlays.getOverlayAtPoint({ x: event.x, y: event.y }); - if (recordIcon === toolBar.clicked(clickedOverlay, false) && !MyAvatar.isPlaying()) { - if (!MyAvatar.isRecording()) { - MyAvatar.startRecording(); + if (recordIcon === toolBar.clicked(clickedOverlay, false) && !Recording.isPlaying()) { + if (!Recording.isRecording()) { + Recording.startRecording(); toolBar.selectTool(recordIcon, false); toolBar.setAlpha(ALPHA_OFF, playIcon); toolBar.setAlpha(ALPHA_OFF, playLoopIcon); toolBar.setAlpha(ALPHA_OFF, saveIcon); toolBar.setAlpha(ALPHA_OFF, loadIcon); } else { - MyAvatar.stopRecording(); + Recording.stopRecording(); toolBar.selectTool(recordIcon, true ); - MyAvatar.loadLastRecording(); + Recording.loadLastRecording(); toolBar.setAlpha(ALPHA_ON, playIcon); toolBar.setAlpha(ALPHA_ON, playLoopIcon); toolBar.setAlpha(ALPHA_ON, saveIcon); toolBar.setAlpha(ALPHA_ON, loadIcon); } - } else if (playIcon === toolBar.clicked(clickedOverlay) && !MyAvatar.isRecording()) { - if (MyAvatar.isPlaying()) { - MyAvatar.pausePlayer(); + } else if (playIcon === toolBar.clicked(clickedOverlay) && !Recording.isRecording()) { + if (Recording.isPlaying()) { + Recording.pausePlayer(); toolBar.setAlpha(ALPHA_ON, recordIcon); toolBar.setAlpha(ALPHA_ON, saveIcon); toolBar.setAlpha(ALPHA_ON, loadIcon); - } else if (MyAvatar.playerLength() > 0) { + } else if (Recording.playerLength() > 0) { setPlayerOptions(); - MyAvatar.setPlayerLoop(false); - MyAvatar.startPlaying(); + Recording.setPlayerLoop(false); + Recording.startPlaying(); toolBar.setAlpha(ALPHA_OFF, recordIcon); toolBar.setAlpha(ALPHA_OFF, saveIcon); toolBar.setAlpha(ALPHA_OFF, loadIcon); watchStop = true; } - } else if (playLoopIcon === toolBar.clicked(clickedOverlay) && !MyAvatar.isRecording()) { - if (MyAvatar.isPlaying()) { - MyAvatar.pausePlayer(); + } else if (playLoopIcon === toolBar.clicked(clickedOverlay) && !Recording.isRecording()) { + if (Recording.isPlaying()) { + Recording.pausePlayer(); toolBar.setAlpha(ALPHA_ON, recordIcon); toolBar.setAlpha(ALPHA_ON, saveIcon); toolBar.setAlpha(ALPHA_ON, loadIcon); - } else if (MyAvatar.playerLength() > 0) { + } else if (Recording.playerLength() > 0) { setPlayerOptions(); - MyAvatar.setPlayerLoop(true); - MyAvatar.startPlaying(); + Recording.setPlayerLoop(true); + Recording.startPlaying(); toolBar.setAlpha(ALPHA_OFF, recordIcon); toolBar.setAlpha(ALPHA_OFF, saveIcon); toolBar.setAlpha(ALPHA_OFF, loadIcon); } } else if (saveIcon === toolBar.clicked(clickedOverlay)) { - if (!MyAvatar.isRecording() && !MyAvatar.isPlaying() && MyAvatar.playerLength() != 0) { + if (!Recording.isRecording() && !Recording.isPlaying() && Recording.playerLength() != 0) { recordingFile = Window.save("Save recording to file", ".", "Recordings (*.hfr)"); if (!(recordingFile === "null" || recordingFile === null || recordingFile === "")) { - MyAvatar.saveRecording(recordingFile); + Recording.saveRecording(recordingFile); } } } else if (loadIcon === toolBar.clicked(clickedOverlay)) { - if (!MyAvatar.isRecording() && !MyAvatar.isPlaying()) { + if (!Recording.isRecording() && !Recording.isPlaying()) { recordingFile = Window.browse("Load recorcding from file", ".", "Recordings (*.hfr *.rec *.HFR *.REC)"); if (!(recordingFile === "null" || recordingFile === null || recordingFile === "")) { - MyAvatar.loadRecording(recordingFile); + Recording.loadRecording(recordingFile); } - if (MyAvatar.playerLength() > 0) { + if (Recording.playerLength() > 0) { toolBar.setAlpha(ALPHA_ON, playIcon); toolBar.setAlpha(ALPHA_ON, playLoopIcon); toolBar.setAlpha(ALPHA_ON, saveIcon); } } - } else if (MyAvatar.playerLength() > 0 && + } else if (Recording.playerLength() > 0 && slider.x < event.x && event.x < slider.x + slider.w && slider.y < event.y && event.y < slider.y + slider.h) { isSliding = true; slider.pos = (event.x - slider.x) / slider.w; - MyAvatar.setPlayerTime(slider.pos * MyAvatar.playerLength()); + Recording.setPlayerTime(slider.pos * Recording.playerLength()); } } var isSliding = false; @@ -296,10 +296,10 @@ function mouseMoveEvent(event) { if (isSliding) { slider.pos = (event.x - slider.x) / slider.w; if (slider.pos < 0.0 || slider.pos > 1.0) { - MyAvatar.stopPlaying(); + Recording.stopPlaying(); slider.pos = 0.0; } - MyAvatar.setPlayerTime(slider.pos * MyAvatar.playerLength()); + Recording.setPlayerTime(slider.pos * Recording.playerLength()); } } @@ -316,7 +316,7 @@ function update() { updateTimer(); - if (watchStop && !MyAvatar.isPlaying()) { + if (watchStop && !Recording.isPlaying()) { watchStop = false; toolBar.setAlpha(ALPHA_ON, recordIcon); toolBar.setAlpha(ALPHA_ON, saveIcon); @@ -325,11 +325,11 @@ function update() { } function scriptEnding() { - if (MyAvatar.isRecording()) { - MyAvatar.stopRecording(); + if (Recording.isRecording()) { + Recording.stopRecording(); } - if (MyAvatar.isPlaying()) { - MyAvatar.stopPlaying(); + if (Recording.isPlaying()) { + Recording.stopPlaying(); } toolBar.cleanup(); Overlays.deleteOverlay(timer); diff --git a/ice-server/src/IceServer.cpp b/ice-server/src/IceServer.cpp index a6a28caa23..f65ff4a8cf 100644 --- a/ice-server/src/IceServer.cpp +++ b/ice-server/src/IceServer.cpp @@ -55,8 +55,7 @@ bool IceServer::packetVersionMatch(const udt::Packet& packet) { if (headerVersion == versionForPacketType(headerType)) { return true; } else { - qDebug() << "Packet version mismatch for packet" << headerType - << "(" << nameForPacketType(headerType) << ") from" << packet.getSenderSockAddr(); + qDebug() << "Packet version mismatch for packet" << headerType << " from" << packet.getSenderSockAddr(); return false; } diff --git a/interface/resources/qml/RecorderDialog.qml b/interface/resources/qml/RecorderDialog.qml new file mode 100644 index 0000000000..4f197846aa --- /dev/null +++ b/interface/resources/qml/RecorderDialog.qml @@ -0,0 +1,105 @@ +// +// Created by Bradley Austin Davis on 2015/11/14 +// Copyright 2015 High Fidelity, Inc. +// +// Distributed under the Apache License, Version 2.0. +// See the accompanying file LICENSE or http://www.apache.org/licenses/LICENSE-2.0.html +// + + +import Hifi 1.0 +import QtQuick 2.4 +import "controls" +import "styles" + +VrDialog { + id: root + HifiConstants { id: hifi } + + property real spacing: hifi.layout.spacing + property real outerSpacing: hifi.layout.spacing * 2 + + objectName: "RecorderDialog" + + destroyOnInvisible: false + destroyOnCloseButton: false + + contentImplicitWidth: recorderDialog.width + contentImplicitHeight: recorderDialog.height + + RecorderDialog { + id: recorderDialog + x: root.clientX; y: root.clientY + width: 800 + height: 128 + signal play() + signal rewind() + + onPlay: { + console.log("Pressed play") + player.isPlaying = !player.isPlaying + } + + onRewind: { + console.log("Pressed rewind") + player.position = 0 + } + + Row { + height: 32 + ButtonAwesome { + id: cmdRecord + visible: root.showRecordButton + width: 32; height: 32 + text: "\uf111" + iconColor: "red" + onClicked: { + console.log("Pressed record") + status.text = "Recording"; + } + } + } + Text { + id: status + anchors.top: parent.top + anchors.right: parent.right + width: 128 + text: "Idle" + } + + Player { + id: player + y: root.clientY + 64 + height: 64 + anchors.left: parent.left + anchors.right: parent.right + anchors.bottom: parent.bottom + + + +// onClicked: { +// if (recordTimer.running) { +// recordTimer.stop(); +// } +// recordTimer.start(); +// } + Timer { + id: recordTimer; + interval: 1000; running: false; repeat: false + onTriggered: { + console.log("Recording: " + MyAvatar.isRecording()) + MyAvatar.startRecording(); + console.log("Recording: " + MyAvatar.isRecording()) + } + } + + } + + Component.onCompleted: { + player.play.connect(play) + player.rewind.connect(rewind) + + } + } +} + diff --git a/interface/resources/qml/controls/ButtonAwesome.qml b/interface/resources/qml/controls/ButtonAwesome.qml new file mode 100644 index 0000000000..47c9fdc742 --- /dev/null +++ b/interface/resources/qml/controls/ButtonAwesome.qml @@ -0,0 +1,21 @@ +import QtQuick 2.3 +import QtQuick.Controls 1.3 as Original +import QtQuick.Controls.Styles 1.3 as OriginalStyles +import "." +import "../styles" + +Original.Button { + property color iconColor: "black" + FontLoader { id: iconFont; source: "../../fonts/fontawesome-webfont.ttf"; } + style: OriginalStyles.ButtonStyle { + label: Text { + renderType: Text.NativeRendering + verticalAlignment: Text.AlignVCenter + horizontalAlignment: Text.AlignHCenter + font.family: iconFont.name + font.pointSize: 20 + color: control.enabled ? control.iconColor : "gray" + text: control.text + } + } +} diff --git a/interface/resources/qml/controls/Player.qml b/interface/resources/qml/controls/Player.qml new file mode 100644 index 0000000000..8af0b1527d --- /dev/null +++ b/interface/resources/qml/controls/Player.qml @@ -0,0 +1,89 @@ +// +// AddressBarDialog.qml +// +// Created by Austin Davis on 2015/04/14 +// Copyright 2015 High Fidelity, Inc. +// +// Distributed under the Apache License, Version 2.0. +// See the accompanying file LICENSE or http://www.apache.org/licenses/LICENSE-2.0.html +// + +//import Hifi 1.0 +import QtQuick 2.4 +import QtQuick.Controls 1.2 +import QtQuick.Dialogs 1.2 +import QtQuick.Controls.Styles 1.2 +import "../styles" + +Item { + id: root + + signal play() + signal rewind() + + property real duration: 100 + property real position: 50 + property bool isPlaying: false + implicitHeight: 64 + implicitWidth: 640 + + Item { + anchors.top: parent.top + anchors.left: parent.left + anchors.right: parent.right + height: root.height / 2 + Text { + id: labelCurrent + anchors.top: parent.top + anchors.bottom: parent.bottom + anchors.left: parent.left + horizontalAlignment: Text.AlignHCenter + verticalAlignment: Text.AlignVCenter + width: 56 + text: "00:00:00" + } + Slider { + value: root.position / root.duration + anchors.top: parent.top + anchors.topMargin: 2 + anchors.bottomMargin: 2 + anchors.bottom: parent.bottom + anchors.left: labelCurrent.right + anchors.leftMargin: 4 + anchors.right: labelDuration.left + anchors.rightMargin: 4 + } + Text { + id: labelDuration + anchors.top: parent.top + anchors.bottom: parent.bottom + anchors.right: parent.right + horizontalAlignment: Text.AlignHCenter + verticalAlignment: Text.AlignVCenter + width: 56 + text: "00:00:00" + } + } + + Row { + anchors.bottom: parent.bottom + anchors.horizontalCenter: parent.horizontalCenter + height: root.height / 2; + ButtonAwesome { + id: cmdPlay + anchors.top: parent.top + anchors.bottom: parent.bottom + text: isPlaying ? "\uf04c" : "\uf04b" + width: root.height / 2; + onClicked: root.play(); + } + ButtonAwesome { + id: cmdRewind + anchors.top: parent.top + anchors.bottom: parent.bottom + width: root.height / 2 + text: "\uf04a" + onClicked: root.rewind(); + } + } +} diff --git a/interface/src/Application.cpp b/interface/src/Application.cpp index 16720fa4cb..3216552719 100644 --- a/interface/src/Application.cpp +++ b/interface/src/Application.cpp @@ -70,6 +70,7 @@ #include #include #include +#include #include #include #include @@ -94,6 +95,8 @@ #include #include #include +#include +#include #include "AnimDebugDraw.h" #include "AudioClient.h" @@ -124,6 +127,7 @@ #include "scripting/LocationScriptingInterface.h" #include "scripting/MenuScriptingInterface.h" #include "scripting/SettingsScriptingInterface.h" +#include "scripting/RecordingScriptingInterface.h" #include "scripting/WebWindowClass.h" #include "scripting/WindowScriptingInterface.h" #include "scripting/ControllerScriptingInterface.h" @@ -132,6 +136,7 @@ #endif #include "Stars.h" #include "ui/AddressBarDialog.h" +#include "ui/RecorderDialog.h" #include "ui/AvatarInputs.h" #include "ui/AssetUploadDialogFactory.h" #include "ui/DataWebDialog.h" @@ -295,6 +300,8 @@ bool setupEssentials(int& argc, char** argv) { Setting::init(); // Set dependencies + DependencyManager::set(); + DependencyManager::set(); DependencyManager::set(); DependencyManager::set(NodeType::Agent, listenPort); DependencyManager::set(); @@ -319,6 +326,7 @@ bool setupEssentials(int& argc, char** argv) { DependencyManager::set(); DependencyManager::set(); DependencyManager::set(); + DependencyManager::set(); DependencyManager::set(); DependencyManager::set(); @@ -332,6 +340,7 @@ bool setupEssentials(int& argc, char** argv) { DependencyManager::set(); DependencyManager::set(); DependencyManager::set(); + DependencyManager::set(); DependencyManager::set(); DependencyManager::set(); return true; @@ -477,6 +486,14 @@ Application::Application(int& argc, char** argv, QElapsedTimer& startupTimer) : connect(assetThread, &QThread::started, assetClient.data(), &AssetClient::init); assetThread->start(); + // Setup MessagesClient + auto messagesClient = DependencyManager::get(); + QThread* messagesThread = new QThread; + messagesThread->setObjectName("Messages Client Thread"); + messagesClient->moveToThread(messagesThread); + connect(messagesThread, &QThread::started, messagesClient.data(), &MessagesClient::init); + messagesThread->start(); + const DomainHandler& domainHandler = nodeList->getDomainHandler(); connect(&domainHandler, SIGNAL(hostnameChanged(const QString&)), SLOT(domainChanged(const QString&))); @@ -543,7 +560,7 @@ Application::Application(int& argc, char** argv, QElapsedTimer& startupTimer) : // tell the NodeList instance who to tell the domain server we care about nodeList->addSetOfNodeTypesToNodeInterestSet(NodeSet() << NodeType::AudioMixer << NodeType::AvatarMixer - << NodeType::EntityServer << NodeType::AssetServer); + << NodeType::EntityServer << NodeType::AssetServer << NodeType::MessagesMixer); // connect to the packet sent signal of the _entityEditSender connect(&_entityEditSender, &EntityEditPacketSender::packetSent, this, &Application::packetSent); @@ -804,8 +821,11 @@ Application::Application(int& argc, char** argv, QElapsedTimer& startupTimer) : void Application::aboutToQuit() { emit beforeAboutToQuit(); + getActiveDisplayPlugin()->deactivate(); + _aboutToQuit = true; + cleanupBeforeQuit(); } @@ -831,8 +851,14 @@ void Application::cleanupBeforeQuit() { _entities.clear(); // this will allow entity scripts to properly shutdown + auto nodeList = DependencyManager::get(); + + // send the domain a disconnect packet, force stoppage of domain-server check-ins + nodeList->getDomainHandler().disconnect(); + nodeList->setIsShuttingDown(true); + // tell the packet receiver we're shutting down, so it can drop packets - DependencyManager::get()->getPacketReceiver().setShouldDropPackets(true); + nodeList->getPacketReceiver().setShouldDropPackets(true); _entities.shutdown(); // tell the entities system we're shutting down, so it will stop running scripts ScriptEngine::stopAllScripts(this); // stop all currently running global scripts @@ -852,9 +878,6 @@ void Application::cleanupBeforeQuit() { saveSettings(); _window->saveGeometry(); - // let the avatar mixer know we're out - MyAvatar::sendKillAvatar(); - // stop the AudioClient QMetaObject::invokeMethod(DependencyManager::get().data(), "stop", Qt::BlockingQueuedConnection); @@ -990,6 +1013,7 @@ void Application::initializeGL() { void Application::initializeUi() { AddressBarDialog::registerType(); + RecorderDialog::registerType(); ErrorDialog::registerType(); LoginDialog::registerType(); MessageDialog::registerType(); @@ -1005,6 +1029,7 @@ void Application::initializeUi() { offscreenUi->load("RootMenu.qml"); auto scriptingInterface = DependencyManager::get(); offscreenUi->getRootContext()->setContextProperty("Controller", scriptingInterface.data()); + offscreenUi->getRootContext()->setContextProperty("MyAvatar", getMyAvatar()); _glWidget->installEventFilter(offscreenUi.data()); VrMenu::load(); VrMenu::executeQueuedLambdas(); @@ -1201,6 +1226,19 @@ void Application::paintGL() { glm::vec3(0.0f, 0.0f, -1.0f) * MIRROR_FULLSCREEN_DISTANCE * _scaleMirror); } renderArgs._renderMode = RenderArgs::MIRROR_RENDER_MODE; + } else if (_myCamera.getMode() == CAMERA_MODE_ENTITY) { + EntityItemPointer cameraEntity = _myCamera.getCameraEntityPointer(); + if (cameraEntity != nullptr) { + if (isHMDMode()) { + glm::quat hmdRotation = extractRotation(myAvatar->getHMDSensorMatrix()); + _myCamera.setRotation(cameraEntity->getRotation() * hmdRotation); + glm::vec3 hmdOffset = extractTranslation(myAvatar->getHMDSensorMatrix()); + _myCamera.setPosition(cameraEntity->getPosition() + (hmdRotation * hmdOffset)); + } else { + _myCamera.setRotation(cameraEntity->getRotation()); + _myCamera.setPosition(cameraEntity->getPosition()); + } + } } // Update camera position if (!isHMDMode()) { @@ -1574,8 +1612,9 @@ void Application::keyPressEvent(QKeyEvent* event) { case Qt::Key_X: if (isMeta && isShifted) { - auto offscreenUi = DependencyManager::get(); - offscreenUi->load("TestControllers.qml"); +// auto offscreenUi = DependencyManager::get(); +// offscreenUi->load("TestControllers.qml"); + RecorderDialog::toggle(); } break; @@ -2644,8 +2683,8 @@ void Application::cycleCamera() { menu->setIsOptionChecked(MenuOption::ThirdPerson, false); menu->setIsOptionChecked(MenuOption::FullscreenMirror, true); - } else if (menu->isOptionChecked(MenuOption::IndependentMode)) { - // do nothing if in independe mode + } else if (menu->isOptionChecked(MenuOption::IndependentMode) || menu->isOptionChecked(MenuOption::CameraEntityMode)) { + // do nothing if in independent or camera entity modes return; } cameraMenuChanged(); // handle the menu change @@ -2672,6 +2711,10 @@ void Application::cameraMenuChanged() { if (_myCamera.getMode() != CAMERA_MODE_INDEPENDENT) { _myCamera.setMode(CAMERA_MODE_INDEPENDENT); } + } else if (Menu::getInstance()->isOptionChecked(MenuOption::CameraEntityMode)) { + if (_myCamera.getMode() != CAMERA_MODE_ENTITY) { + _myCamera.setMode(CAMERA_MODE_ENTITY); + } } } @@ -3957,6 +4000,7 @@ void Application::registerScriptEngineWithApplicationServices(ScriptEngine* scri RayToOverlayIntersectionResultFromScriptValue); scriptEngine->registerGlobalObject("Desktop", DependencyManager::get().data()); + scriptEngine->registerGlobalObject("Recording", DependencyManager::get().data()); scriptEngine->registerGlobalObject("Window", DependencyManager::get().data()); scriptEngine->registerGetterSetter("location", LocationScriptingInterface::locationGetter, diff --git a/interface/src/Camera.cpp b/interface/src/Camera.cpp index 17c745bdba..53a3500bff 100644 --- a/interface/src/Camera.cpp +++ b/interface/src/Camera.cpp @@ -28,6 +28,8 @@ CameraMode stringToMode(const QString& mode) { return CAMERA_MODE_MIRROR; } else if (mode == "independent") { return CAMERA_MODE_INDEPENDENT; + } else if (mode == "entity") { + return CAMERA_MODE_ENTITY; } return CAMERA_MODE_NULL; } @@ -41,6 +43,8 @@ QString modeToString(CameraMode mode) { return "mirror"; } else if (mode == CAMERA_MODE_INDEPENDENT) { return "independent"; + } else if (mode == CAMERA_MODE_ENTITY) { + return "entity"; } return "unknown"; } @@ -94,6 +98,17 @@ void Camera::setMode(CameraMode mode) { emit modeUpdated(modeToString(mode)); } +QUuid Camera::getCameraEntity() const { + if (_cameraEntity != nullptr) { + return _cameraEntity->getID(); + } + return QUuid(); +}; + +void Camera::setCameraEntity(QUuid entityID) { + _cameraEntity = qApp->getEntities()->getTree()->findEntityByID(entityID); +} + void Camera::setProjection(const glm::mat4& projection) { _projection = projection; } @@ -118,6 +133,9 @@ void Camera::setModeString(const QString& mode) { case CAMERA_MODE_INDEPENDENT: Menu::getInstance()->setIsOptionChecked(MenuOption::IndependentMode, true); break; + case CAMERA_MODE_ENTITY: + Menu::getInstance()->setIsOptionChecked(MenuOption::CameraEntityMode, true); + break; default: break; } diff --git a/interface/src/Camera.h b/interface/src/Camera.h index 7f7515cf5f..017bd742a4 100644 --- a/interface/src/Camera.h +++ b/interface/src/Camera.h @@ -24,6 +24,7 @@ enum CameraMode CAMERA_MODE_FIRST_PERSON, CAMERA_MODE_MIRROR, CAMERA_MODE_INDEPENDENT, + CAMERA_MODE_ENTITY, NUM_CAMERA_MODES }; @@ -36,6 +37,7 @@ class Camera : public QObject { Q_PROPERTY(glm::vec3 position READ getPosition WRITE setPosition) Q_PROPERTY(glm::quat orientation READ getOrientation WRITE setOrientation) Q_PROPERTY(QString mode READ getModeString WRITE setModeString) + Q_PROPERTY(QUuid cameraEntity READ getCameraEntity WRITE setCameraEntity) public: Camera(); @@ -49,6 +51,8 @@ public: void loadViewFrustum(ViewFrustum& frustum) const; ViewFrustum toViewFrustum() const; + EntityItemPointer getCameraEntityPointer() const { return _cameraEntity; } + public slots: QString getModeString() const; void setModeString(const QString& mode); @@ -68,6 +72,9 @@ public slots: const glm::mat4& getProjection() const { return _projection; } void setProjection(const glm::mat4& projection); + QUuid getCameraEntity() const; + void setCameraEntity(QUuid entityID); + PickRay computePickRay(float x, float y); // These only work on independent cameras @@ -97,6 +104,7 @@ private: glm::quat _rotation; bool _isKeepLookingAt{ false }; glm::vec3 _lookingAt; + EntityItemPointer _cameraEntity; }; #endif // hifi_Camera_h diff --git a/interface/src/Menu.cpp b/interface/src/Menu.cpp index b17aca94fb..a6db205fe1 100644 --- a/interface/src/Menu.cpp +++ b/interface/src/Menu.cpp @@ -266,6 +266,9 @@ Menu::Menu() { cameraModeGroup->addAction(addCheckableActionToQMenuAndActionHash(cameraModeMenu, MenuOption::IndependentMode, 0, false, qApp, SLOT(cameraMenuChanged()))); + cameraModeGroup->addAction(addCheckableActionToQMenuAndActionHash(cameraModeMenu, + MenuOption::CameraEntityMode, 0, + false, qApp, SLOT(cameraMenuChanged()))); cameraModeGroup->addAction(addCheckableActionToQMenuAndActionHash(cameraModeMenu, MenuOption::FullscreenMirror, 0, // QML Qt::Key_H, false, qApp, SLOT(cameraMenuChanged()))); diff --git a/interface/src/Menu.h b/interface/src/Menu.h index dfa2cfa41b..7546ed59f2 100644 --- a/interface/src/Menu.h +++ b/interface/src/Menu.h @@ -155,6 +155,7 @@ namespace MenuOption { const QString Bookmarks = "Bookmarks"; const QString CachesSize = "RAM Caches Size"; const QString CalibrateCamera = "Calibrate Camera"; + const QString CameraEntityMode = "Entity Mode"; const QString CenterPlayerInView = "Center Player In View"; const QString Chat = "Chat..."; const QString Collisions = "Collisions"; diff --git a/interface/src/avatar/AvatarActionHold.cpp b/interface/src/avatar/AvatarActionHold.cpp index 839f2d4fbb..8e13fa8385 100644 --- a/interface/src/avatar/AvatarActionHold.cpp +++ b/interface/src/avatar/AvatarActionHold.cpp @@ -10,7 +10,6 @@ // #include "QVariantGLM.h" -#include "avatar/MyAvatar.h" #include "avatar/AvatarManager.h" #include "AvatarActionHold.h" @@ -22,8 +21,7 @@ AvatarActionHold::AvatarActionHold(const QUuid& id, EntityItemPointer ownerEntit _relativePosition(glm::vec3(0.0f)), _relativeRotation(glm::quat()), _hand("right"), - _holderID(QUuid()) -{ + _holderID(QUuid()) { _type = ACTION_TYPE_HOLD; #if WANT_DEBUG qDebug() << "AvatarActionHold::AvatarActionHold"; @@ -36,13 +34,10 @@ AvatarActionHold::~AvatarActionHold() { #endif } -void AvatarActionHold::updateActionWorker(float deltaTimeStep) { - bool gotLock = false; - glm::quat rotation; - glm::vec3 position; +std::shared_ptr AvatarActionHold::getTarget(glm::quat& rotation, glm::vec3& position) { std::shared_ptr holdingAvatar = nullptr; - gotLock = withTryReadLock([&]{ + withTryReadLock([&]{ QSharedPointer avatarManager = DependencyManager::get(); AvatarSharedPointer holdingAvatarData = avatarManager->getAvatarBySessionID(_holderID); holdingAvatar = std::static_pointer_cast(holdingAvatarData); @@ -65,22 +60,53 @@ void AvatarActionHold::updateActionWorker(float deltaTimeStep) { } }); - if (holdingAvatar) { + return holdingAvatar; +} + +void AvatarActionHold::updateActionWorker(float deltaTimeStep) { + glm::quat rotation; + glm::vec3 position; + bool valid = false; + int holdCount = 0; + + auto ownerEntity = _ownerEntity.lock(); + if (!ownerEntity) { + return; + } + QList holdActions = ownerEntity->getActionsOfType(ACTION_TYPE_HOLD); + foreach (EntityActionPointer action, holdActions) { + std::shared_ptr holdAction = std::static_pointer_cast(action); + glm::quat rotationForAction; + glm::vec3 positionForAction; + std::shared_ptr holdingAvatar = holdAction->getTarget(rotationForAction, positionForAction); + if (holdingAvatar) { + holdCount ++; + if (holdAction.get() == this) { + // only use the rotation for this action + valid = true; + rotation = rotationForAction; + } + + position += positionForAction; + } + } + + if (valid && holdCount > 0) { + position /= holdCount; + + bool gotLock = withTryWriteLock([&]{ + _positionalTarget = position; + _rotationalTarget = rotation; + _positionalTargetSet = true; + _rotationalTargetSet = true; + _active = true; + }); if (gotLock) { - gotLock = withTryWriteLock([&]{ - _positionalTarget = position; - _rotationalTarget = rotation; - _positionalTargetSet = true; - _rotationalTargetSet = true; - _active = true; - }); - if (gotLock) { - if (_kinematic) { - doKinematicUpdate(deltaTimeStep); - } else { - activateBody(); - ObjectActionSpring::updateActionWorker(deltaTimeStep); - } + if (_kinematic) { + doKinematicUpdate(deltaTimeStep); + } else { + activateBody(); + ObjectActionSpring::updateActionWorker(deltaTimeStep); } } } @@ -109,7 +135,8 @@ void AvatarActionHold::doKinematicUpdate(float deltaTimeStep) { if (_previousSet) { // smooth velocity over 2 frames glm::vec3 positionalDelta = _positionalTarget - _previousPositionalTarget; - glm::vec3 positionalVelocity = (positionalDelta + _previousPositionalDelta) / (deltaTimeStep + _previousDeltaTimeStep); + glm::vec3 positionalVelocity = + (positionalDelta + _previousPositionalDelta) / (deltaTimeStep + _previousDeltaTimeStep); rigidBody->setLinearVelocity(glmToBullet(positionalVelocity)); _previousPositionalDelta = positionalDelta; _previousDeltaTimeStep = deltaTimeStep; diff --git a/interface/src/avatar/AvatarActionHold.h b/interface/src/avatar/AvatarActionHold.h index 15a096d1ce..b8b1a64e84 100644 --- a/interface/src/avatar/AvatarActionHold.h +++ b/interface/src/avatar/AvatarActionHold.h @@ -17,6 +17,9 @@ #include #include +#include "avatar/MyAvatar.h" + + class AvatarActionHold : public ObjectActionSpring { public: AvatarActionHold(const QUuid& id, EntityItemPointer ownerEntity); @@ -32,6 +35,8 @@ public: virtual bool shouldSuppressLocationEdits() { return _active && !_ownerEntity.expired(); } + std::shared_ptr getTarget(glm::quat& rotation, glm::vec3& position); + private: static const uint16_t holdVersion; diff --git a/interface/src/avatar/Head.cpp b/interface/src/avatar/Head.cpp index 1a9b8a49e2..e8452583fc 100644 --- a/interface/src/avatar/Head.cpp +++ b/interface/src/avatar/Head.cpp @@ -14,6 +14,7 @@ #include #include #include +#include #include "Application.h" #include "Avatar.h" @@ -90,10 +91,9 @@ void Head::simulate(float deltaTime, bool isMine, bool billboard) { } if (isMine) { - MyAvatar* myAvatar = static_cast(_owningAvatar); - + auto player = DependencyManager::get(); // Only use face trackers when not playing back a recording. - if (!myAvatar->isPlaying()) { + if (!player->isPlaying()) { FaceTracker* faceTracker = qApp->getActiveFaceTracker(); _isFaceTrackerConnected = faceTracker != NULL && !faceTracker->isMuted(); if (_isFaceTrackerConnected) { diff --git a/interface/src/avatar/MyAvatar.cpp b/interface/src/avatar/MyAvatar.cpp index eafc50ef33..5bacb567e1 100644 --- a/interface/src/avatar/MyAvatar.cpp +++ b/interface/src/avatar/MyAvatar.cpp @@ -79,10 +79,6 @@ const QString& DEFAULT_AVATAR_COLLISION_SOUND_URL = "https://hifi-public.s3.amaz const float MyAvatar::ZOOM_MIN = 0.5f; const float MyAvatar::ZOOM_MAX = 25.0f; const float MyAvatar::ZOOM_DEFAULT = 1.5f; -static const QString HEADER_NAME = "com.highfidelity.recording.AvatarData"; -static recording::FrameType AVATAR_FRAME_TYPE = recording::Frame::TYPE_INVALID; -static std::once_flag frameTypeRegistration; - MyAvatar::MyAvatar(RigPointer rig) : Avatar(rig), @@ -120,17 +116,6 @@ MyAvatar::MyAvatar(RigPointer rig) : { using namespace recording; - std::call_once(frameTypeRegistration, [] { - AVATAR_FRAME_TYPE = recording::Frame::registerFrameType(HEADER_NAME); - }); - - // FIXME how to deal with driving multiple avatars locally? - Frame::registerFrameHandler(AVATAR_FRAME_TYPE, [this](Frame::ConstPointer frame) { - qDebug() << "Playback of avatar frame length: " << frame->data.size(); - avatarStateFromFrame(frame->data, this); - }); - - for (int i = 0; i < MAX_DRIVE_KEYS; i++) { _driveKeys[i] = 0.0f; } @@ -169,9 +154,8 @@ void MyAvatar::reset(bool andReload) { // Reset dynamic state. _wasPushing = _isPushing = _isBraking = _billboardValid = false; - _isFollowingHMD = false; - _hmdFollowVelocity = Vectors::ZERO; - _hmdFollowSpeed = 0.0f; + _followVelocity = Vectors::ZERO; + _followSpeed = 0.0f; _skeletonModel.reset(); getHead()->reset(); _targetVelocity = glm::vec3(0.0f); @@ -306,8 +290,10 @@ void MyAvatar::simulate(float deltaTime) { } // Record avatars movements. - if (_recorder && _recorder->isRecording()) { - _recorder->recordFrame(AVATAR_FRAME_TYPE, avatarStateToFrame(this)); + auto recorder = DependencyManager::get(); + if (recorder->isRecording()) { + static const recording::FrameType FRAME_TYPE = recording::Frame::registerFrameType(AvatarData::FRAME_NAME); + recorder->recordFrame(FRAME_TYPE, toFrame(*this)); } // consider updating our billboard @@ -331,52 +317,40 @@ void MyAvatar::updateFromHMDSensorMatrix(const glm::mat4& hmdSensorMatrix) { void MyAvatar::updateHMDFollowVelocity() { // compute offset to body's target position (in sensor-frame) auto sensorBodyMatrix = deriveBodyFromHMDSensor(); - _hmdFollowOffset = extractTranslation(sensorBodyMatrix) - extractTranslation(_bodySensorMatrix); - glm::vec3 truncatedOffset = _hmdFollowOffset; - if (truncatedOffset.y < 0.0f) { - // don't pull the body DOWN to match the target (allow animation system to squat) - truncatedOffset.y = 0.0f; - } - float truncatedOffsetDistance = glm::length(truncatedOffset); + glm::vec3 offset = extractTranslation(sensorBodyMatrix) - extractTranslation(_bodySensorMatrix); + _followOffsetDistance = glm::length(offset); + + const float FOLLOW_TIMESCALE = 0.5f; + const float FOLLOW_THRESHOLD_SPEED = 0.2f; + const float FOLLOW_MIN_DISTANCE = 0.01f; + const float FOLLOW_THRESHOLD_DISTANCE = 0.2f; + const float FOLLOW_MAX_IDLE_DISTANCE = 0.1f; - bool isMoving; - if (_lastIsMoving) { - const float MOVE_EXIT_SPEED_THRESHOLD = 0.07f; // m/sec - isMoving = glm::length(_velocity) >= MOVE_EXIT_SPEED_THRESHOLD; - } else { - const float MOVE_ENTER_SPEED_THRESHOLD = 0.2f; // m/sec - isMoving = glm::length(_velocity) > MOVE_ENTER_SPEED_THRESHOLD; - } - bool justStartedMoving = (_lastIsMoving != isMoving) && isMoving; - _lastIsMoving = isMoving; bool hmdIsAtRest = _hmdAtRestDetector.update(_hmdSensorPosition, _hmdSensorOrientation); - const float MIN_HMD_HIP_SHIFT = 0.05f; - if (justStartedMoving || (hmdIsAtRest && truncatedOffsetDistance > MIN_HMD_HIP_SHIFT)) { - _isFollowingHMD = true; - } - bool needNewFollowSpeed = (_isFollowingHMD && _hmdFollowSpeed == 0.0f); - if (!needNewFollowSpeed) { - // check to see if offset has exceeded its threshold - const float MAX_HMD_HIP_SHIFT = 0.2f; - if (truncatedOffsetDistance > MAX_HMD_HIP_SHIFT) { - _isFollowingHMD = true; - needNewFollowSpeed = true; + _followOffsetDistance = glm::length(offset); + if (_followOffsetDistance < FOLLOW_MIN_DISTANCE) { + // close enough + _followOffsetDistance = 0.0f; + } else { + bool avatarIsMoving = glm::length(_velocity - _followVelocity) > FOLLOW_THRESHOLD_SPEED; + bool shouldFollow = (hmdIsAtRest || avatarIsMoving) && _followOffsetDistance > FOLLOW_MAX_IDLE_DISTANCE; + + glm::vec3 truncatedOffset = offset; + if (truncatedOffset.y < 0.0f) { + truncatedOffset.y = 0.0f; + } + float truncatedDistance = glm::length(truncatedOffset); + bool needsNewSpeed = truncatedDistance > FOLLOW_THRESHOLD_DISTANCE; + if (needsNewSpeed || (shouldFollow && _followSpeed == 0.0f)) { + // compute new speed + _followSpeed = _followOffsetDistance / FOLLOW_TIMESCALE; + } + if (_followSpeed > 0.0f) { + // to compute new velocity we must rotate offset into the world-frame + glm::quat sensorToWorldRotation = extractRotation(_sensorToWorldMatrix); + _followVelocity = _followSpeed * glm::normalize(sensorToWorldRotation * offset); } - } - if (_isFollowingHMD) { - // only bother to rotate into world frame if we're following - glm::quat sensorToWorldRotation = extractRotation(_sensorToWorldMatrix); - _hmdFollowOffset = sensorToWorldRotation * _hmdFollowOffset; - } - if (needNewFollowSpeed) { - // compute new velocity that will be used to resolve offset of hips from body - const float FOLLOW_HMD_DURATION = 0.5f; // seconds - _hmdFollowVelocity = (_hmdFollowOffset / FOLLOW_HMD_DURATION); - _hmdFollowSpeed = glm::length(_hmdFollowVelocity); - } else if (_isFollowingHMD) { - // compute new velocity (but not new speed) - _hmdFollowVelocity = _hmdFollowSpeed * glm::normalize(_hmdFollowOffset); } } @@ -384,6 +358,13 @@ void MyAvatar::updateHMDFollowVelocity() { // update sensor to world matrix from current body position and hmd sensor. // This is so the correct camera can be used for rendering. void MyAvatar::updateSensorToWorldMatrix() { + +#ifdef DEBUG_RENDERING + // draw marker about avatar's position + const glm::vec4 red(1.0f, 0.0f, 0.0f, 1.0f); + DebugDraw::getInstance().addMyAvatarMarker("pos", glm::quat(), glm::vec3(), red); +#endif + // update the sensor mat so that the body position will end up in the desired // position when driven from the head. glm::mat4 desiredMat = createMatFromQuatAndPos(getOrientation(), getPosition()); @@ -395,8 +376,8 @@ void MyAvatar::updateFromTrackers(float deltaTime) { glm::vec3 estimatedPosition, estimatedRotation; bool inHmd = qApp->getAvatarUpdater()->isHMDMode(); - - if (isPlaying() && inHmd) { + bool playing = DependencyManager::get()->isPlaying(); + if (inHmd && playing) { return; } @@ -447,7 +428,7 @@ void MyAvatar::updateFromTrackers(float deltaTime) { Head* head = getHead(); - if (inHmd || isPlaying()) { + if (inHmd || playing) { head->setDeltaPitch(estimatedRotation.x); head->setDeltaYaw(estimatedRotation.y); head->setDeltaRoll(estimatedRotation.z); @@ -564,102 +545,6 @@ bool MyAvatar::setJointReferential(const QUuid& id, int jointIndex) { } } -bool MyAvatar::isRecording() { - if (!_recorder) { - return false; - } - if (QThread::currentThread() != thread()) { - bool result; - QMetaObject::invokeMethod(this, "isRecording", Qt::BlockingQueuedConnection, - Q_RETURN_ARG(bool, result)); - return result; - } - return _recorder && _recorder->isRecording(); -} - -float MyAvatar::recorderElapsed() { - if (QThread::currentThread() != thread()) { - float result; - QMetaObject::invokeMethod(this, "recorderElapsed", Qt::BlockingQueuedConnection, - Q_RETURN_ARG(float, result)); - return result; - } - if (!_recorder) { - return 0; - } - return (float)_recorder->position() / (float) MSECS_PER_SECOND; -} - -QMetaObject::Connection _audioClientRecorderConnection; - -void MyAvatar::startRecording() { - if (QThread::currentThread() != thread()) { - QMetaObject::invokeMethod(this, "startRecording", Qt::BlockingQueuedConnection); - return; - } - - _recorder = std::make_shared(); - // connect to AudioClient's signal so we get input audio - auto audioClient = DependencyManager::get(); - _audioClientRecorderConnection = connect(audioClient.data(), &AudioClient::inputReceived, [] { - // FIXME, missing audio data handling - }); - setRecordingBasis(); - _recorder->start(); -} - -void MyAvatar::stopRecording() { - if (!_recorder) { - return; - } - if (QThread::currentThread() != thread()) { - QMetaObject::invokeMethod(this, "stopRecording", Qt::BlockingQueuedConnection); - return; - } - if (_recorder) { - QObject::disconnect(_audioClientRecorderConnection); - _audioClientRecorderConnection = QMetaObject::Connection(); - _recorder->stop(); - clearRecordingBasis(); - } -} - -void MyAvatar::saveRecording(const QString& filename) { - if (!_recorder) { - qCDebug(interfaceapp) << "There is no recording to save"; - return; - } - if (QThread::currentThread() != thread()) { - QMetaObject::invokeMethod(this, "saveRecording", Qt::BlockingQueuedConnection, - Q_ARG(QString, filename)); - return; - } - - if (_recorder) { - auto clip = _recorder->getClip(); - recording::Clip::toFile(filename, clip); - } -} - -void MyAvatar::loadLastRecording() { - if (QThread::currentThread() != thread()) { - QMetaObject::invokeMethod(this, "loadLastRecording", Qt::BlockingQueuedConnection); - return; - } - - if (!_recorder || !_recorder->getClip()) { - qCDebug(interfaceapp) << "There is no recording to load"; - return; - } - - if (!_player) { - _player = std::make_shared(); - } - - _player->queueClip(_recorder->getClip()); - _player->play(); -} - void MyAvatar::overrideAnimation(const QString& url, float fps, bool loop, float firstFrame, float lastFrame) { if (QThread::currentThread() != thread()) { QMetaObject::invokeMethod(this, "overrideAnimation", Q_ARG(const QString&, url), Q_ARG(float, fps), @@ -899,11 +784,6 @@ int MyAvatar::parseDataFromBuffer(const QByteArray& buffer) { return buffer.size(); } -void MyAvatar::sendKillAvatar() { - auto killPacket = NLPacket::create(PacketType::KillAvatar, 0); - DependencyManager::get()->broadcastToNodes(std::move(killPacket), NodeSet() << NodeType::AvatarMixer); -} - void MyAvatar::updateLookAtTargetAvatar() { // // Look at the avatar whose eyes are closest to the ray in direction of my avatar's head @@ -1160,11 +1040,11 @@ void MyAvatar::prepareForPhysicsSimulation() { _characterController.setAvatarPositionAndOrientation(getPosition(), getOrientation()); if (qApp->isHMDMode()) { updateHMDFollowVelocity(); - } else if (_isFollowingHMD) { - _isFollowingHMD = false; - _hmdFollowVelocity = Vectors::ZERO; + } else if (_followSpeed > 0.0f) { + _followVelocity = Vectors::ZERO; + _followSpeed = 0.0f; } - _characterController.setHMDVelocity(_hmdFollowVelocity); + _characterController.setFollowVelocity(_followVelocity); } void MyAvatar::harvestResultsFromPhysicsSimulation() { @@ -1172,35 +1052,27 @@ void MyAvatar::harvestResultsFromPhysicsSimulation() { glm::quat orientation = getOrientation(); _characterController.getAvatarPositionAndOrientation(position, orientation); nextAttitude(position, orientation); - if (_isFollowingHMD) { - setVelocity(_characterController.getLinearVelocity() + _hmdFollowVelocity); - glm::vec3 hmdShift = _characterController.getHMDShift(); - adjustSensorTransform(hmdShift); + if (_followSpeed > 0.0f) { + adjustSensorTransform(); + setVelocity(_characterController.getLinearVelocity() + _followVelocity); } else { setVelocity(_characterController.getLinearVelocity()); } } -void MyAvatar::adjustSensorTransform(glm::vec3 hmdShift) { +void MyAvatar::adjustSensorTransform() { // compute blendFactor of latest hmdShift // which we'll use to blend the rotation part - float blendFactor = 1.0f; - float shiftLength = glm::length(hmdShift); - if (shiftLength > 1.0e-5f) { - float offsetLength = glm::length(_hmdFollowOffset); - if (offsetLength > shiftLength) { - blendFactor = shiftLength / offsetLength; - } - } + float linearDistance = _characterController.getFollowTime() * _followSpeed; + float blendFactor = linearDistance < _followOffsetDistance ? linearDistance / _followOffsetDistance : 1.0f; auto newBodySensorMatrix = deriveBodyFromHMDSensor(); auto worldBodyMatrix = _sensorToWorldMatrix * newBodySensorMatrix; glm::quat finalBodyRotation = glm::normalize(glm::quat_cast(worldBodyMatrix)); if (blendFactor >= 0.99f) { // the "adjustment" is more or less complete so stop following - _isFollowingHMD = false; - _hmdFollowSpeed = 0.0f; - _hmdFollowVelocity = Vectors::ZERO; + _followVelocity = Vectors::ZERO; + _followSpeed = 0.0f; // and slam the body's transform anyway to eliminate any slight errors glm::vec3 finalBodyPosition = extractTranslation(worldBodyMatrix); nextAttitude(finalBodyPosition, finalBodyRotation); @@ -1379,6 +1251,9 @@ void MyAvatar::initAnimGraph() { QUrl::fromLocalFile(PathUtils::resourcesPath() + "meshes/defaultAvatar_full/avatar-animation.json") : _animGraphUrl); _rig->initAnimGraph(graphUrl); + + _bodySensorMatrix = deriveBodyFromHMDSensor(); // Based on current cached HMD position/rotation.. + updateSensorToWorldMatrix(); // Uses updated position/orientation and _bodySensorMatrix changes } void MyAvatar::destroyAnimGraph() { @@ -1850,8 +1725,26 @@ glm::quat MyAvatar::getWorldBodyOrientation() const { return glm::quat_cast(_sensorToWorldMatrix * _bodySensorMatrix); } +#if 0 // derive avatar body position and orientation from the current HMD Sensor location. // results are in sensor space +glm::mat4 MyAvatar::deriveBodyFromHMDSensor() const { + if (_rig) { + // orientation + const glm::quat hmdOrientation = getHMDSensorOrientation(); + const glm::quat yaw = cancelOutRollAndPitch(hmdOrientation); + // position + // we flip about yAxis when going from "root" to "avatar" frame + // and we must also apply "yaw" to get into HMD frame + glm::quat rotY180 = glm::angleAxis((float)M_PI, glm::vec3(0.0f, 1.0f, 0.0f)); + glm::vec3 eyesInAvatarFrame = rotY180 * yaw * _rig->getEyesInRootFrame(); + glm::vec3 bodyPos = getHMDSensorPosition() - eyesInAvatarFrame; + return createMatFromQuatAndPos(yaw, bodyPos); + } + return glm::mat4(); +} +#else +// old school meat hook style glm::mat4 MyAvatar::deriveBodyFromHMDSensor() const { // HMD is in sensor space. @@ -1859,10 +1752,18 @@ glm::mat4 MyAvatar::deriveBodyFromHMDSensor() const { const glm::quat hmdOrientation = getHMDSensorOrientation(); const glm::quat hmdOrientationYawOnly = cancelOutRollAndPitch(hmdOrientation); + /* const glm::vec3 DEFAULT_RIGHT_EYE_POS(-0.3f, 1.6f, 0.0f); const glm::vec3 DEFAULT_LEFT_EYE_POS(0.3f, 1.6f, 0.0f); const glm::vec3 DEFAULT_NECK_POS(0.0f, 1.5f, 0.0f); const glm::vec3 DEFAULT_HIPS_POS(0.0f, 1.0f, 0.0f); + */ + + // 2 meter tall dude + const glm::vec3 DEFAULT_RIGHT_EYE_POS(-0.3f, 1.9f, 0.0f); + const glm::vec3 DEFAULT_LEFT_EYE_POS(0.3f, 1.9f, 0.0f); + const glm::vec3 DEFAULT_NECK_POS(0.0f, 1.70f, 0.0f); + const glm::vec3 DEFAULT_HIPS_POS(0.0f, 1.05f, 0.0f); vec3 localEyes, localNeck; if (!_debugDrawSkeleton) { @@ -1901,6 +1802,7 @@ glm::mat4 MyAvatar::deriveBodyFromHMDSensor() const { // avatar facing is determined solely by hmd orientation. return createMatFromQuatAndPos(hmdOrientationYawOnly, bodyPos); } +#endif glm::vec3 MyAvatar::getPositionForAudio() { switch (_audioListenerMode) { diff --git a/interface/src/avatar/MyAvatar.h b/interface/src/avatar/MyAvatar.h index c8b7b7bf02..e6578c4e13 100644 --- a/interface/src/avatar/MyAvatar.h +++ b/interface/src/avatar/MyAvatar.h @@ -159,8 +159,6 @@ public: eyeContactTarget getEyeContactTarget(); - static void sendKillAvatar(); - Q_INVOKABLE glm::vec3 getTrackedHeadPosition() const { return _trackedHeadPosition; } Q_INVOKABLE glm::vec3 getHeadPosition() const { return getHead()->getPosition(); } Q_INVOKABLE float getHeadFinalYaw() const { return getHead()->getFinalYaw(); } @@ -205,7 +203,7 @@ public: void prepareForPhysicsSimulation(); void harvestResultsFromPhysicsSimulation(); - void adjustSensorTransform(glm::vec3 hmdShift); + void adjustSensorTransform(); const QString& getCollisionSoundURL() { return _collisionSoundURL; } void setCollisionSoundURL(const QString& url); @@ -253,13 +251,6 @@ public slots: bool setModelReferential(const QUuid& id); bool setJointReferential(const QUuid& id, int jointIndex); - bool isRecording(); - float recorderElapsed(); - void startRecording(); - void stopRecording(); - void saveRecording(const QString& filename); - void loadLastRecording(); - virtual void rebuildSkeletonBody() override; const QString& getAnimGraphUrl() const { return _animGraphUrl; } @@ -305,9 +296,6 @@ private: const glm::vec3& translation = glm::vec3(), const glm::quat& rotation = glm::quat(), float scale = 1.0f, bool allowDuplicates = false, bool useSaved = true) override; - const recording::RecorderPointer getRecorder() const { return _recorder; } - const recording::DeckPointer getPlayer() const { return _player; } - //void beginFollowingHMD(); //bool shouldFollowHMD() const; //void followHMD(float deltaTime); @@ -324,7 +312,7 @@ private: PalmData getActivePalmData(int palmIndex) const; // derive avatar body position and orientation from the current HMD Sensor location. - // results are in sensor space + // results are in HMD frame glm::mat4 deriveBodyFromHMDSensor() const; float _driveKeys[MAX_DRIVE_KEYS]; @@ -354,8 +342,6 @@ private: eyeContactTarget _eyeContactTarget; - recording::RecorderPointer _recorder; - glm::vec3 _trackedHeadPosition; Setting::Handle _realWorldFieldOfView; @@ -387,9 +373,10 @@ private: // used to transform any sensor into world space, including the _hmdSensorMat, or hand controllers. glm::mat4 _sensorToWorldMatrix; - glm::vec3 _hmdFollowOffset { Vectors::ZERO }; - glm::vec3 _hmdFollowVelocity { Vectors::ZERO }; - float _hmdFollowSpeed { 0.0f }; + + glm::vec3 _followVelocity { Vectors::ZERO }; + float _followSpeed { 0.0f }; + float _followOffsetDistance { 0.0f }; bool _goToPending; glm::vec3 _goToPosition; @@ -407,9 +394,6 @@ private: glm::vec3 _customListenPosition; glm::quat _customListenOrientation; - bool _isFollowingHMD { false }; - float _followHMDAlpha { 0.0f }; - AtRestDetector _hmdAtRestDetector; bool _lastIsMoving { false }; }; diff --git a/interface/src/avatar/MyCharacterController.cpp b/interface/src/avatar/MyCharacterController.cpp index ad2ca32b05..e8f686da6f 100644 --- a/interface/src/avatar/MyCharacterController.cpp +++ b/interface/src/avatar/MyCharacterController.cpp @@ -60,7 +60,7 @@ MyCharacterController::MyCharacterController(MyAvatar* avatar) { _floorDistance = MAX_FALL_HEIGHT; _walkVelocity.setValue(0.0f, 0.0f, 0.0f); - _hmdVelocity.setValue(0.0f, 0.0f, 0.0f); + _followVelocity.setValue(0.0f, 0.0f, 0.0f); _jumpSpeed = JUMP_SPEED; _isOnGround = false; _isJumping = false; @@ -68,7 +68,7 @@ MyCharacterController::MyCharacterController(MyAvatar* avatar) { _isHovering = true; _isPushingUp = false; _jumpToHoverStart = 0; - _lastStepDuration = 0.0f; + _followTime = 0.0f; _pendingFlags = PENDING_FLAG_UPDATE_SHAPE; updateShapeIfNecessary(); @@ -161,16 +161,14 @@ void MyCharacterController::playerStep(btCollisionWorld* dynaWorld, btScalar dt) } } - // Rather than add _hmdVelocity to the velocity of the RigidBody, we explicitly teleport + // Rather than add _followVelocity to the velocity of the RigidBody, we explicitly teleport // the RigidBody forward according to the formula: distance = rate * time - if (_hmdVelocity.length2() > 0.0f) { + if (_followVelocity.length2() > 0.0f) { btTransform bodyTransform = _rigidBody->getWorldTransform(); - bodyTransform.setOrigin(bodyTransform.getOrigin() + dt * _hmdVelocity); + bodyTransform.setOrigin(bodyTransform.getOrigin() + dt * _followVelocity); _rigidBody->setWorldTransform(bodyTransform); } - // MyAvatar will ask us how far we stepped for HMD motion, which will depend on how - // much time has accumulated in _lastStepDuration. - _lastStepDuration += dt; + _followTime += dt; } void MyCharacterController::jump() { @@ -346,8 +344,8 @@ void MyCharacterController::setTargetVelocity(const glm::vec3& velocity) { _walkVelocity = glmToBullet(velocity); } -void MyCharacterController::setHMDVelocity(const glm::vec3& velocity) { - _hmdVelocity = glmToBullet(velocity); +void MyCharacterController::setFollowVelocity(const glm::vec3& velocity) { + _followVelocity = glmToBullet(velocity); } glm::vec3 MyCharacterController::getLinearVelocity() const { @@ -400,7 +398,7 @@ void MyCharacterController::preSimulation() { } } } - _lastStepDuration = 0.0f; + _followTime = 0.0f; } void MyCharacterController::postSimulation() { diff --git a/interface/src/avatar/MyCharacterController.h b/interface/src/avatar/MyCharacterController.h index de711c84f4..82aa958309 100644 --- a/interface/src/avatar/MyCharacterController.h +++ b/interface/src/avatar/MyCharacterController.h @@ -64,8 +64,8 @@ public: void getAvatarPositionAndOrientation(glm::vec3& position, glm::quat& rotation) const; void setTargetVelocity(const glm::vec3& velocity); - void setHMDVelocity(const glm::vec3& velocity); - glm::vec3 getHMDShift() const { return _lastStepDuration * bulletToGLM(_hmdVelocity); } + void setFollowVelocity(const glm::vec3& velocity); + float getFollowTime() const { return _followTime; } glm::vec3 getLinearVelocity() const; @@ -75,7 +75,7 @@ protected: protected: btVector3 _currentUp; btVector3 _walkVelocity; - btVector3 _hmdVelocity; + btVector3 _followVelocity; btTransform _avatarBodyTransform; glm::vec3 _shapeLocalOffset; @@ -93,7 +93,7 @@ protected: btScalar _gravity; btScalar _jumpSpeed; - btScalar _lastStepDuration; + btScalar _followTime; bool _enabled; bool _isOnGround; diff --git a/interface/src/avatar/SkeletonModel.cpp b/interface/src/avatar/SkeletonModel.cpp index dcdb6d4dcb..8b34d9248e 100644 --- a/interface/src/avatar/SkeletonModel.cpp +++ b/interface/src/avatar/SkeletonModel.cpp @@ -13,6 +13,7 @@ #include #include +#include #include "Application.h" #include "Avatar.h" @@ -249,9 +250,8 @@ void SkeletonModel::simulate(float deltaTime, bool fullUpdate) { return; // only simulate for own avatar } - MyAvatar* myAvatar = static_cast(_owningAvatar); - if (myAvatar->isPlaying()) { - // Don't take inputs if playing back a recording. + auto player = DependencyManager::get(); + if (player->isPlaying()) { return; } } diff --git a/interface/src/scripting/RecordingScriptingInterface.cpp b/interface/src/scripting/RecordingScriptingInterface.cpp new file mode 100644 index 0000000000..d549de84b2 --- /dev/null +++ b/interface/src/scripting/RecordingScriptingInterface.cpp @@ -0,0 +1,255 @@ +// +// Created by Bradley Austin Davis on 2015/11/13 +// Copyright 2015 High Fidelity, Inc. +// +// Distributed under the Apache License, Version 2.0. +// See the accompanying file LICENSE or http://www.apache.org/licenses/LICENSE-2.0.html +// + +#include "RecordingScriptingInterface.h" + +#include +#include +#include +#include +#include +#include +#include + +#include "avatar/AvatarManager.h" +#include "Application.h" +#include "InterfaceLogging.h" + +typedef int16_t AudioSample; + + +using namespace recording; + +// FIXME move to somewhere audio related? +static const QString AUDIO_FRAME_NAME = "com.highfidelity.recording.Audio"; + +RecordingScriptingInterface::RecordingScriptingInterface() { + static const recording::FrameType AVATAR_FRAME_TYPE = recording::Frame::registerFrameType(AvatarData::FRAME_NAME); + // FIXME how to deal with driving multiple avatars locally? + Frame::registerFrameHandler(AVATAR_FRAME_TYPE, [this](Frame::ConstPointer frame) { + processAvatarFrame(frame); + }); + + static const recording::FrameType AUDIO_FRAME_TYPE = recording::Frame::registerFrameType(AUDIO_FRAME_NAME); + Frame::registerFrameHandler(AUDIO_FRAME_TYPE, [this](Frame::ConstPointer frame) { + processAudioFrame(frame); + }); + + _player = DependencyManager::get(); + _recorder = DependencyManager::get(); + + auto audioClient = DependencyManager::get(); + connect(audioClient.data(), &AudioClient::inputReceived, this, &RecordingScriptingInterface::processAudioInput); +} + +bool RecordingScriptingInterface::isPlaying() const { + return _player->isPlaying(); +} + +bool RecordingScriptingInterface::isPaused() const { + return _player->isPaused(); +} + +float RecordingScriptingInterface::playerElapsed() const { + return _player->position(); +} + +float RecordingScriptingInterface::playerLength() const { + return _player->length(); +} + +void RecordingScriptingInterface::loadRecording(const QString& filename) { + using namespace recording; + + if (QThread::currentThread() != thread()) { + QMetaObject::invokeMethod(this, "loadRecording", Qt::BlockingQueuedConnection, + Q_ARG(QString, filename)); + return; + } + + ClipPointer clip = Clip::fromFile(filename); + if (!clip) { + qWarning() << "Unable to load clip data from " << filename; + } + _player->queueClip(clip); +} + +void RecordingScriptingInterface::startPlaying() { + if (QThread::currentThread() != thread()) { + QMetaObject::invokeMethod(this, "startPlaying", Qt::BlockingQueuedConnection); + return; + } + auto myAvatar = DependencyManager::get()->getMyAvatar(); + // Playback from the current position + if (_playFromCurrentLocation) { + _dummyAvatar.setRecordingBasis(std::make_shared(myAvatar->getTransform())); + } else { + _dummyAvatar.clearRecordingBasis(); + } + _player->play(); +} + +void RecordingScriptingInterface::setPlayerVolume(float volume) { + // FIXME +} + +void RecordingScriptingInterface::setPlayerAudioOffset(float audioOffset) { + // FIXME +} + +void RecordingScriptingInterface::setPlayerTime(float time) { + if (QThread::currentThread() != thread()) { + QMetaObject::invokeMethod(this, "setPlayerTime", Qt::BlockingQueuedConnection, Q_ARG(float, time)); + return; + } + _player->seek(time); +} + +void RecordingScriptingInterface::setPlayFromCurrentLocation(bool playFromCurrentLocation) { + _playFromCurrentLocation = playFromCurrentLocation; +} + +void RecordingScriptingInterface::setPlayerLoop(bool loop) { + _player->loop(loop); +} + +void RecordingScriptingInterface::setPlayerUseDisplayName(bool useDisplayName) { + _useDisplayName = useDisplayName; +} + +void RecordingScriptingInterface::setPlayerUseAttachments(bool useAttachments) { + _useAttachments = useAttachments; +} + +void RecordingScriptingInterface::setPlayerUseHeadModel(bool useHeadModel) { + _useHeadModel = useHeadModel; +} + +void RecordingScriptingInterface::setPlayerUseSkeletonModel(bool useSkeletonModel) { + _useSkeletonModel = useSkeletonModel; +} + +void RecordingScriptingInterface::pausePlayer() { + if (QThread::currentThread() != thread()) { + QMetaObject::invokeMethod(this, "pausePlayer", Qt::BlockingQueuedConnection); + return; + } + _player->pause(); +} + +void RecordingScriptingInterface::stopPlaying() { + if (QThread::currentThread() != thread()) { + QMetaObject::invokeMethod(this, "stopPlaying", Qt::BlockingQueuedConnection); + return; + } + _player->stop(); +} + +bool RecordingScriptingInterface::isRecording() const { + return _recorder->isRecording(); +} + +float RecordingScriptingInterface::recorderElapsed() const { + return _recorder->position(); +} + +void RecordingScriptingInterface::startRecording() { + if (_recorder->isRecording()) { + qCWarning(interfaceapp) << "Recorder is already running"; + return; + } + + if (QThread::currentThread() != thread()) { + QMetaObject::invokeMethod(this, "startRecording", Qt::BlockingQueuedConnection); + return; + } + + _recordingEpoch = Frame::epochForFrameTime(0); + DependencyManager::get()->getMyAvatar()->setRecordingBasis(); + _recorder->start(); +} + +void RecordingScriptingInterface::stopRecording() { + _recorder->stop(); + _lastClip = _recorder->getClip(); + _lastClip->seek(0); + DependencyManager::get()->getMyAvatar()->clearRecordingBasis(); +} + +void RecordingScriptingInterface::saveRecording(const QString& filename) { + if (QThread::currentThread() != thread()) { + QMetaObject::invokeMethod(this, "saveRecording", Qt::BlockingQueuedConnection, + Q_ARG(QString, filename)); + return; + } + + if (!_lastClip) { + qWarning() << "There is no recording to save"; + return; + } + + recording::Clip::toFile(filename, _lastClip); +} + +void RecordingScriptingInterface::loadLastRecording() { + if (QThread::currentThread() != thread()) { + QMetaObject::invokeMethod(this, "loadLastRecording", Qt::BlockingQueuedConnection); + return; + } + + if (!_lastClip) { + qCDebug(interfaceapp) << "There is no recording to load"; + return; + } + + _player->queueClip(_lastClip); + _player->play(); +} + +void RecordingScriptingInterface::processAvatarFrame(const Frame::ConstPointer& frame) { + Q_ASSERT(QThread::currentThread() == thread()); + + AvatarData::fromFrame(frame->data, _dummyAvatar); + + auto myAvatar = DependencyManager::get()->getMyAvatar(); + if (_useHeadModel && _dummyAvatar.getFaceModelURL().isValid() && + (_dummyAvatar.getFaceModelURL() != myAvatar->getFaceModelURL())) { + // FIXME + //myAvatar->setFaceModelURL(_dummyAvatar.getFaceModelURL()); + } + + if (_useSkeletonModel && _dummyAvatar.getSkeletonModelURL().isValid() && + (_dummyAvatar.getSkeletonModelURL() != myAvatar->getSkeletonModelURL())) { + // FIXME + //myAvatar->useFullAvatarURL() + } + + if (_useDisplayName && _dummyAvatar.getDisplayName() != myAvatar->getDisplayName()) { + myAvatar->setDisplayName(_dummyAvatar.getDisplayName()); + } + + myAvatar->setPosition(_dummyAvatar.getPosition()); + myAvatar->setOrientation(_dummyAvatar.getOrientation()); + + // FIXME attachments + // FIXME joints + // FIXME head lean + // FIXME head orientation +} + +void RecordingScriptingInterface::processAudioInput(const QByteArray& audio) { + if (_recorder->isRecording()) { + static const recording::FrameType AUDIO_FRAME_TYPE = recording::Frame::registerFrameType(AUDIO_FRAME_NAME); + _recorder->recordFrame(AUDIO_FRAME_TYPE, audio); + } +} + +void RecordingScriptingInterface::processAudioFrame(const recording::FrameConstPointer& frame) { + auto audioClient = DependencyManager::get(); + audioClient->handleRecordedAudioInput(frame->data); +} diff --git a/interface/src/scripting/RecordingScriptingInterface.h b/interface/src/scripting/RecordingScriptingInterface.h new file mode 100644 index 0000000000..27193e6f9a --- /dev/null +++ b/interface/src/scripting/RecordingScriptingInterface.h @@ -0,0 +1,84 @@ +// +// Created by Bradley Austin Davis on 2015/11/13 +// Copyright 2015 High Fidelity, Inc. +// +// Distributed under the Apache License, Version 2.0. +// See the accompanying file LICENSE or http://www.apache.org/licenses/LICENSE-2.0.html +// + +#ifndef hifi_RecordingScriptingInterface_h +#define hifi_RecordingScriptingInterface_h + +#include + +#include + +#include +#include +#include +#include + +class RecordingScriptingInterface : public QObject, public Dependency { + Q_OBJECT + +public: + RecordingScriptingInterface(); + +public slots: + void loadRecording(const QString& filename); + + void startPlaying(); + void pausePlayer(); + void stopPlaying(); + bool isPlaying() const; + bool isPaused() const; + + float playerElapsed() const; + float playerLength() const; + + void setPlayerVolume(float volume); + void setPlayerAudioOffset(float audioOffset); + void setPlayerTime(float time); + void setPlayFromCurrentLocation(bool playFromCurrentLocation); + void setPlayerLoop(bool loop); + void setPlayerUseDisplayName(bool useDisplayName); + void setPlayerUseAttachments(bool useAttachments); + void setPlayerUseHeadModel(bool useHeadModel); + void setPlayerUseSkeletonModel(bool useSkeletonModel); + + void startRecording(); + void stopRecording(); + bool isRecording() const; + + float recorderElapsed() const; + + void saveRecording(const QString& filename); + void loadLastRecording(); + +signals: + void playbackStateChanged(); + // Should this occur for any frame or just for seek calls? + void playbackPositionChanged(); + void looped(); + +private: + using Mutex = std::recursive_mutex; + using Locker = std::unique_lock; + using Flag = std::atomic; + void processAvatarFrame(const recording::FrameConstPointer& frame); + void processAudioFrame(const recording::FrameConstPointer& frame); + void processAudioInput(const QByteArray& audioData); + QSharedPointer _player; + QSharedPointer _recorder; + quint64 _recordingEpoch { 0 }; + + Flag _playFromCurrentLocation { true }; + Flag _useDisplayName { false }; + Flag _useHeadModel { false }; + Flag _useAttachments { false }; + Flag _useSkeletonModel { false }; + recording::ClipPointer _lastClip; + AvatarData _dummyAvatar; +}; + +#endif // hifi_RecordingScriptingInterface_h diff --git a/interface/src/ui/RecorderDialog.cpp b/interface/src/ui/RecorderDialog.cpp new file mode 100644 index 0000000000..ddefa9fbd9 --- /dev/null +++ b/interface/src/ui/RecorderDialog.cpp @@ -0,0 +1,22 @@ +// +// Created by Bradley Austin Davis on 2015/11/14 +// Copyright 2015 High Fidelity, Inc. +// +// Distributed under the Apache License, Version 2.0. +// See the accompanying file LICENSE or http://www.apache.org/licenses/LICENSE-2.0.html +// + +#include "RecorderDialog.h" + +#include + +#include "DependencyManager.h" + +HIFI_QML_DEF(RecorderDialog) + +RecorderDialog::RecorderDialog(QQuickItem* parent) : OffscreenQmlDialog(parent) { +} + +void RecorderDialog::hide() { + ((QQuickItem*)parent())->setEnabled(false); +} diff --git a/interface/src/ui/RecorderDialog.h b/interface/src/ui/RecorderDialog.h new file mode 100644 index 0000000000..f4f0a7c2d8 --- /dev/null +++ b/interface/src/ui/RecorderDialog.h @@ -0,0 +1,28 @@ +// +// Created by Bradley Austin Davis on 2015/11/14 +// Copyright 2015 High Fidelity, Inc. +// +// Distributed under the Apache License, Version 2.0. +// See the accompanying file LICENSE or http://www.apache.org/licenses/LICENSE-2.0.html +// + +#pragma once +#ifndef hifi_RecorderDialog_h +#define hifi_RecorderDialog_h + +#include + +class RecorderDialog : public OffscreenQmlDialog { + Q_OBJECT + HIFI_QML_DECL + +public: + RecorderDialog(QQuickItem* parent = nullptr); + +signals: + +protected: + void hide(); +}; + +#endif diff --git a/libraries/animation/src/AnimSkeleton.cpp b/libraries/animation/src/AnimSkeleton.cpp index 0db7473c9c..7ec8db1490 100644 --- a/libraries/animation/src/AnimSkeleton.cpp +++ b/libraries/animation/src/AnimSkeleton.cpp @@ -79,7 +79,7 @@ const QString& AnimSkeleton::getJointName(int jointIndex) const { } AnimPose AnimSkeleton::getAbsolutePose(int jointIndex, const AnimPoseVec& poses) const { - if (jointIndex < 0) { + if (jointIndex < 0 || jointIndex >= (int)poses.size() || jointIndex >= (int)_joints.size()) { return AnimPose::identity; } else { return getAbsolutePose(_joints[jointIndex].parentIndex, poses) * poses[jointIndex]; diff --git a/libraries/animation/src/Rig.cpp b/libraries/animation/src/Rig.cpp index 71e7a27d47..00f93a5d5e 100644 --- a/libraries/animation/src/Rig.cpp +++ b/libraries/animation/src/Rig.cpp @@ -35,6 +35,19 @@ } while (0) #endif +/* +const glm::vec3 DEFAULT_RIGHT_EYE_POS(-0.3f, 1.6f, 0.0f); +const glm::vec3 DEFAULT_LEFT_EYE_POS(0.3f, 1.6f, 0.0f); +const glm::vec3 DEFAULT_HEAD_POS(0.0f, 1.55f, 0.0f); +const glm::vec3 DEFAULT_NECK_POS(0.0f, 1.5f, 0.0f); +*/ + +// 2 meter tall dude +const glm::vec3 DEFAULT_RIGHT_EYE_POS(-0.3f, 1.9f, 0.0f); +const glm::vec3 DEFAULT_LEFT_EYE_POS(0.3f, 1.9f, 0.0f); +const glm::vec3 DEFAULT_HEAD_POS(0.0f, 1.75f, 0.0f); +const glm::vec3 DEFAULT_NECK_POS(0.0f, 1.70f, 0.0f); + void Rig::overrideAnimation(const QString& url, float fps, bool loop, float firstFrame, float lastFrame) { // find an unused AnimClip clipNode @@ -155,6 +168,7 @@ void Rig::initJointStates(const FBXGeometry& geometry, glm::mat4 modelOffset, in int rightHandJointIndex, int rightElbowJointIndex, int rightShoulderJointIndex) { _animSkeleton = std::make_shared(geometry); + computeEyesInRootFrame(_animSkeleton->getRelativeBindPoses()); _relativePoses.clear(); _relativePoses = _animSkeleton->getRelativeBindPoses(); @@ -436,6 +450,26 @@ void Rig::calcAnimAlpha(float speed, const std::vector& referenceSpeeds, *alphaOut = alpha; } +void Rig::computeEyesInRootFrame(const AnimPoseVec& poses) { + // TODO: use cached eye/hips indices for these calculations + int numPoses = poses.size(); + int hipsIndex = _animSkeleton->nameToJointIndex(QString("Hips")); + int headIndex = _animSkeleton->nameToJointIndex(QString("Head")); + if (hipsIndex > 0 && headIndex > 0) { + int rightEyeIndex = _animSkeleton->nameToJointIndex(QString("RightEye")); + int leftEyeIndex = _animSkeleton->nameToJointIndex(QString("LeftEye")); + if (numPoses > rightEyeIndex && numPoses > leftEyeIndex && rightEyeIndex > 0 && leftEyeIndex > 0) { + glm::vec3 rightEye = _animSkeleton->getAbsolutePose(rightEyeIndex, poses).trans; + glm::vec3 leftEye = _animSkeleton->getAbsolutePose(leftEyeIndex, poses).trans; + glm::vec3 hips = _animSkeleton->getAbsolutePose(hipsIndex, poses).trans; + _eyesInRootFrame = 0.5f * (rightEye + leftEye) - hips; + } else { + glm::vec3 hips = _animSkeleton->getAbsolutePose(hipsIndex, poses).trans; + _eyesInRootFrame = 0.5f * (DEFAULT_RIGHT_EYE_POS + DEFAULT_LEFT_EYE_POS) - hips; + } + } +} + // animation reference speeds. static const std::vector FORWARD_SPEEDS = { 0.4f, 1.4f, 4.5f }; // m/s static const std::vector BACKWARD_SPEEDS = { 0.6f, 1.45f }; // m/s @@ -721,6 +755,8 @@ void Rig::updateAnimations(float deltaTime, glm::mat4 rootTransform) { setJointTranslation((int)i, true, _relativePoses[i].trans, PRIORITY); } } + + computeEyesInRootFrame(_relativePoses); } setModelOffset(rootTransform); @@ -975,14 +1011,14 @@ void Rig::updateLeanJoint(int index, float leanSideways, float leanForward, floa static AnimPose avatarToBonePose(AnimPose pose, AnimSkeleton::ConstPointer skeleton) { AnimPose rootPose = skeleton->getAbsoluteBindPose(skeleton->nameToJointIndex("Hips")); - AnimPose rotY180(glm::vec3(1), glm::angleAxis((float)PI, glm::vec3(0.0f, 1.0f, 0.0f)), glm::vec3(0)); + AnimPose rotY180(glm::vec3(1.0f), glm::angleAxis(PI, glm::vec3(0.0f, 1.0f, 0.0f)), glm::vec3(0)); return rootPose * rotY180 * pose; } #ifdef DEBUG_RENDERING static AnimPose boneToAvatarPose(AnimPose pose, AnimSkeleton::ConstPointer skeleton) { AnimPose rootPose = skeleton->getAbsoluteBindPose(skeleton->nameToJointIndex("Hips")); - AnimPose rotY180(glm::vec3(1), glm::angleAxis((float)PI, glm::vec3(0.0f, 1.0f, 0.0f)), glm::vec3(0)); + AnimPose rotY180(glm::vec3(1.0f), glm::angleAxis(PI, glm::vec3(0.0f, 1.0f, 0.0f)), glm::vec3(0)); return (rootPose * rotY180).inverse() * pose; } #endif @@ -1004,11 +1040,6 @@ static void computeHeadNeckAnimVars(AnimSkeleton::ConstPointer skeleton, const A int headIndex = skeleton->nameToJointIndex("Head"); int neckIndex = skeleton->nameToJointIndex("Neck"); - const glm::vec3 DEFAULT_RIGHT_EYE_POS(-0.3f, 1.6f, 0.0f); - const glm::vec3 DEFAULT_LEFT_EYE_POS(0.3f, 1.6f, 0.0f); - const glm::vec3 DEFAULT_HEAD_POS(0.0f, 1.55f, 0.0f); - const glm::vec3 DEFAULT_NECK_POS(0.0f, 1.5f, 0.0f); - // Use absolute bindPose positions just in case the relBindPose have rotations we don't expect. glm::vec3 absRightEyePos = rightEyeIndex != -1 ? skeleton->getAbsoluteBindPose(rightEyeIndex).trans : DEFAULT_RIGHT_EYE_POS; glm::vec3 absLeftEyePos = leftEyeIndex != -1 ? skeleton->getAbsoluteBindPose(leftEyeIndex).trans : DEFAULT_LEFT_EYE_POS; diff --git a/libraries/animation/src/Rig.h b/libraries/animation/src/Rig.h index 09ec7c0e2f..d6bbee7ebf 100644 --- a/libraries/animation/src/Rig.h +++ b/libraries/animation/src/Rig.h @@ -164,6 +164,8 @@ public: bool getModelOffset(glm::vec3& modelOffsetOut) const; + const glm::vec3& getEyesInRootFrame() const { return _eyesInRootFrame; } + protected: void updateAnimationStateHandlers(); void applyOverridePoses(); @@ -174,6 +176,8 @@ public: void updateEyeJoint(int index, const glm::vec3& modelTranslation, const glm::quat& modelRotation, const glm::quat& worldHeadOrientation, const glm::vec3& lookAt, const glm::vec3& saccade); void calcAnimAlpha(float speed, const std::vector& referenceSpeeds, float* alphaOut) const; + void computeEyesInRootFrame(const AnimPoseVec& poses); + // AJT: TODO: LEGACY QVector _jointStates; glm::mat4 _legacyModelOffset; @@ -198,6 +202,7 @@ public: glm::vec3 _lastFront; glm::vec3 _lastPosition; glm::vec3 _lastVelocity; + glm::vec3 _eyesInRootFrame { Vectors::ZERO }; std::shared_ptr _animNode; std::shared_ptr _animSkeleton; diff --git a/libraries/audio-client/src/AudioClient.cpp b/libraries/audio-client/src/AudioClient.cpp index 72e47073f2..d4e571ade5 100644 --- a/libraries/audio-client/src/AudioClient.cpp +++ b/libraries/audio-client/src/AudioClient.cpp @@ -743,19 +743,9 @@ void AudioClient::handleLocalEchoAndReverb(QByteArray& inputByteArray) { } void AudioClient::handleAudioInput() { - if (!_audioPacket) { - // we don't have an audioPacket yet - set that up now - _audioPacket = NLPacket::create(PacketType::MicrophoneAudioNoEcho); - } - const float inputToNetworkInputRatio = calculateDeviceToNetworkInputRatio(); - const int inputSamplesRequired = (int)((float)AudioConstants::NETWORK_FRAME_SAMPLES_PER_CHANNEL * inputToNetworkInputRatio); const auto inputAudioSamples = std::unique_ptr(new int16_t[inputSamplesRequired]); - - static const int leadingBytes = sizeof(quint16) + sizeof(glm::vec3) + sizeof(glm::quat) + sizeof(quint8); - int16_t* const networkAudioSamples = (int16_t*)(_audioPacket->getPayload() + leadingBytes); - QByteArray inputByteArray = _inputDevice->readAll(); // Add audio source injection if enabled @@ -784,30 +774,30 @@ void AudioClient::handleAudioInput() { float audioInputMsecsRead = inputByteArray.size() / (float)(_inputFormat.bytesForDuration(USECS_PER_MSEC)); _stats.updateInputMsecsRead(audioInputMsecsRead); - while (_inputRingBuffer.samplesAvailable() >= inputSamplesRequired) { + const int numNetworkBytes = _isStereoInput + ? AudioConstants::NETWORK_FRAME_BYTES_STEREO + : AudioConstants::NETWORK_FRAME_BYTES_PER_CHANNEL; + const int numNetworkSamples = _isStereoInput + ? AudioConstants::NETWORK_FRAME_SAMPLES_STEREO + : AudioConstants::NETWORK_FRAME_SAMPLES_PER_CHANNEL; - const int numNetworkBytes = _isStereoInput - ? AudioConstants::NETWORK_FRAME_BYTES_STEREO - : AudioConstants::NETWORK_FRAME_BYTES_PER_CHANNEL; - const int numNetworkSamples = _isStereoInput - ? AudioConstants::NETWORK_FRAME_SAMPLES_STEREO - : AudioConstants::NETWORK_FRAME_SAMPLES_PER_CHANNEL; + static int16_t networkAudioSamples[AudioConstants::NETWORK_FRAME_SAMPLES_STEREO]; + + while (_inputRingBuffer.samplesAvailable() >= inputSamplesRequired) { if (!_muted) { - // zero out the monoAudioSamples array and the locally injected audio - memset(networkAudioSamples, 0, numNetworkBytes); // Increment the time since the last clip if (_timeSinceLastClip >= 0.0f) { - _timeSinceLastClip += (float) numNetworkSamples / (float) AudioConstants::SAMPLE_RATE; + _timeSinceLastClip += (float)numNetworkSamples / (float)AudioConstants::SAMPLE_RATE; } _inputRingBuffer.readSamples(inputAudioSamples.get(), inputSamplesRequired); possibleResampling(_inputToNetworkResampler, - inputAudioSamples.get(), networkAudioSamples, - inputSamplesRequired, numNetworkSamples, - _inputFormat, _desiredInputFormat); + inputAudioSamples.get(), networkAudioSamples, + inputSamplesRequired, numNetworkSamples, + _inputFormat, _desiredInputFormat); // Remove DC offset if (!_isStereoInput && !_audioSourceInjectEnabled) { @@ -829,7 +819,7 @@ void AudioClient::handleAudioInput() { for (int i = 0; i < numNetworkSamples; i++) { int thisSample = std::abs(networkAudioSamples[i]); - loudness += (float) thisSample; + loudness += (float)thisSample; if (thisSample > (AudioConstants::MAX_SAMPLE_VALUE * AudioNoiseGate::CLIPPING_THRESHOLD)) { _timeSinceLastClip = 0.0f; @@ -839,7 +829,7 @@ void AudioClient::handleAudioInput() { _lastInputLoudness = fabs(loudness / numNetworkSamples); } - emit inputReceived({reinterpret_cast(networkAudioSamples), numNetworkBytes}); + emit inputReceived({ reinterpret_cast(networkAudioSamples), numNetworkBytes }); } else { // our input loudness is 0, since we're muted @@ -849,14 +839,38 @@ void AudioClient::handleAudioInput() { _inputRingBuffer.shiftReadPosition(inputSamplesRequired); } - auto nodeList = DependencyManager::get(); - SharedNodePointer audioMixer = nodeList->soloNodeOfType(NodeType::AudioMixer); + emitAudioPacket(networkAudioSamples); + } +} - if (audioMixer && audioMixer->getActiveSocket()) { - glm::vec3 headPosition = _positionGetter(); - glm::quat headOrientation = _orientationGetter(); - quint8 isStereo = _isStereoInput ? 1 : 0; +void AudioClient::emitAudioPacket(const int16_t* audioData, PacketType packetType) { + static std::mutex _mutex; + using Locker = std::unique_lock; + // FIXME recorded audio isn't guaranteed to have the same stereo state + // as the current system + const int numNetworkBytes = _isStereoInput + ? AudioConstants::NETWORK_FRAME_BYTES_STEREO + : AudioConstants::NETWORK_FRAME_BYTES_PER_CHANNEL; + const int numNetworkSamples = _isStereoInput + ? AudioConstants::NETWORK_FRAME_SAMPLES_STEREO + : AudioConstants::NETWORK_FRAME_SAMPLES_PER_CHANNEL; + + auto nodeList = DependencyManager::get(); + SharedNodePointer audioMixer = nodeList->soloNodeOfType(NodeType::AudioMixer); + + if (audioMixer && audioMixer->getActiveSocket()) { + Locker lock(_mutex); + if (!_audioPacket) { + // we don't have an audioPacket yet - set that up now + _audioPacket = NLPacket::create(PacketType::MicrophoneAudioWithEcho); + } + + glm::vec3 headPosition = _positionGetter(); + glm::quat headOrientation = _orientationGetter(); + quint8 isStereo = _isStereoInput ? 1 : 0; + + if (packetType == PacketType::Unknown) { if (_lastInputLoudness == 0) { _audioPacket->setType(PacketType::SilentAudioFrame); } else { @@ -866,44 +880,54 @@ void AudioClient::handleAudioInput() { _audioPacket->setType(PacketType::MicrophoneAudioNoEcho); } } - - // reset the audio packet so we can start writing - _audioPacket->reset(); - - // write sequence number - _audioPacket->writePrimitive(_outgoingAvatarAudioSequenceNumber); - - if (_audioPacket->getType() == PacketType::SilentAudioFrame) { - // pack num silent samples - quint16 numSilentSamples = numNetworkSamples; - _audioPacket->writePrimitive(numSilentSamples); - } else { - // set the mono/stereo byte - _audioPacket->writePrimitive(isStereo); - } - - // pack the three float positions - _audioPacket->writePrimitive(headPosition); - - // pack the orientation - _audioPacket->writePrimitive(headOrientation); - - if (_audioPacket->getType() != PacketType::SilentAudioFrame) { - // audio samples have already been packed (written to networkAudioSamples) - _audioPacket->setPayloadSize(_audioPacket->getPayloadSize() + numNetworkBytes); - } - - _stats.sentPacket(); - - nodeList->flagTimeForConnectionStep(LimitedNodeList::ConnectionStep::SendAudioPacket); - - nodeList->sendUnreliablePacket(*_audioPacket, *audioMixer); - - _outgoingAvatarAudioSequenceNumber++; + } else { + _audioPacket->setType(packetType); } + + // reset the audio packet so we can start writing + _audioPacket->reset(); + + // write sequence number + _audioPacket->writePrimitive(_outgoingAvatarAudioSequenceNumber); + + if (_audioPacket->getType() == PacketType::SilentAudioFrame) { + // pack num silent samples + quint16 numSilentSamples = numNetworkSamples; + _audioPacket->writePrimitive(numSilentSamples); + } else { + // set the mono/stereo byte + _audioPacket->writePrimitive(isStereo); + } + + // pack the three float positions + _audioPacket->writePrimitive(headPosition); + + // pack the orientation + _audioPacket->writePrimitive(headOrientation); + + if (_audioPacket->getType() != PacketType::SilentAudioFrame) { + // audio samples have already been packed (written to networkAudioSamples) + _audioPacket->setPayloadSize(_audioPacket->getPayloadSize() + numNetworkBytes); + } + + static const int leadingBytes = sizeof(quint16) + sizeof(glm::vec3) + sizeof(glm::quat) + sizeof(quint8); + int16_t* const networkAudioSamples = (int16_t*)(_audioPacket->getPayload() + leadingBytes); + memcpy(networkAudioSamples, audioData, numNetworkBytes); + + _stats.sentPacket(); + + nodeList->flagTimeForConnectionStep(LimitedNodeList::ConnectionStep::SendAudioPacket); + + nodeList->sendUnreliablePacket(*_audioPacket, *audioMixer); + + _outgoingAvatarAudioSequenceNumber++; } } +void AudioClient::handleRecordedAudioInput(const QByteArray& audio) { + emitAudioPacket((int16_t*)audio.data(), PacketType::MicrophoneAudioWithEcho); +} + void AudioClient::processReceivedSamples(const QByteArray& inputBuffer, QByteArray& outputBuffer) { const int numNetworkOutputSamples = inputBuffer.size() / sizeof(int16_t); const int numDeviceOutputSamples = numNetworkOutputSamples * (_outputFormat.sampleRate() * _outputFormat.channelCount()) diff --git a/libraries/audio-client/src/AudioClient.h b/libraries/audio-client/src/AudioClient.h index e699ee9266..9d46ad9d26 100644 --- a/libraries/audio-client/src/AudioClient.h +++ b/libraries/audio-client/src/AudioClient.h @@ -147,6 +147,7 @@ public slots: void sendDownstreamAudioStatsPacket() { _stats.sendDownstreamAudioStatsPacket(); } void handleAudioInput(); + void handleRecordedAudioInput(const QByteArray& audio); void reset(); void audioMixerKilled(); void toggleMute(); @@ -211,6 +212,7 @@ protected: } private: + void emitAudioPacket(const int16_t* audioData, PacketType packetType = PacketType::Unknown); void outputFormatChanged(); QByteArray firstInputFrame; diff --git a/libraries/avatars/CMakeLists.txt b/libraries/avatars/CMakeLists.txt index 6d4d9cc341..fc6d15cced 100644 --- a/libraries/avatars/CMakeLists.txt +++ b/libraries/avatars/CMakeLists.txt @@ -1,3 +1,3 @@ set(TARGET_NAME avatars) setup_hifi_library(Network Script) -link_hifi_libraries(audio shared networking recording) +link_hifi_libraries(shared networking) diff --git a/libraries/avatars/src/AvatarData.cpp b/libraries/avatars/src/AvatarData.cpp index a47d5f663e..44a781bc24 100644 --- a/libraries/avatars/src/AvatarData.cpp +++ b/libraries/avatars/src/AvatarData.cpp @@ -33,8 +33,6 @@ #include #include #include -#include -#include #include "AvatarLogging.h" @@ -45,6 +43,9 @@ using namespace std; const glm::vec3 DEFAULT_LOCAL_AABOX_CORNER(-0.5f); const glm::vec3 DEFAULT_LOCAL_AABOX_SCALE(1.0f); +const QString AvatarData::FRAME_NAME = "com.highfidelity.recording.AvatarData"; +static std::once_flag frameTypeRegistration; + AvatarData::AvatarData() : _sessionUUID(), _position(0.0f), @@ -176,7 +177,7 @@ float AvatarData::getTargetScale() const { void AvatarData::setTargetScale(float targetScale, bool overideReferential) { if (!_referential || overideReferential) { - _targetScale = targetScale; + _targetScale = std::max(MIN_AVATAR_SCALE, std::min(MAX_AVATAR_SCALE, targetScale)); } } @@ -530,7 +531,7 @@ int AvatarData::parseDataFromBuffer(const QByteArray& buffer) { } return maxAvailableSize; } - _targetScale = scale; + _targetScale = std::max(MIN_AVATAR_SCALE, std::min(MAX_AVATAR_SCALE, scale)); } // 20 bytes { // Lookat Position @@ -791,155 +792,10 @@ bool AvatarData::hasReferential() { return _referential != NULL; } -bool AvatarData::isPlaying() { - return _player && _player->isPlaying(); -} - -bool AvatarData::isPaused() { - return _player && _player->isPaused(); -} - -float AvatarData::playerElapsed() { - if (!_player) { - return 0; - } - if (QThread::currentThread() != thread()) { - float result; - QMetaObject::invokeMethod(this, "playerElapsed", Qt::BlockingQueuedConnection, - Q_RETURN_ARG(float, result)); - return result; - } - return (float)_player->position() / (float) MSECS_PER_SECOND; -} - -float AvatarData::playerLength() { - if (!_player) { - return 0; - } - if (QThread::currentThread() != thread()) { - float result; - QMetaObject::invokeMethod(this, "playerLength", Qt::BlockingQueuedConnection, - Q_RETURN_ARG(float, result)); - return result; - } - return (float)_player->length() / (float) MSECS_PER_SECOND; -} - -void AvatarData::loadRecording(const QString& filename) { - if (QThread::currentThread() != thread()) { - QMetaObject::invokeMethod(this, "loadRecording", Qt::BlockingQueuedConnection, - Q_ARG(QString, filename)); - return; - } - using namespace recording; - - ClipPointer clip = Clip::fromFile(filename); - if (!clip) { - qWarning() << "Unable to load clip data from " << filename; - } - - _player = std::make_shared(); - _player->queueClip(clip); -} - -void AvatarData::startPlaying() { - if (QThread::currentThread() != thread()) { - QMetaObject::invokeMethod(this, "startPlaying", Qt::BlockingQueuedConnection); - return; - } - - if (!_player) { - qWarning() << "No clip loaded for playback"; - return; - } - setRecordingBasis(); - _player->play(); -} - -void AvatarData::setPlayerVolume(float volume) { - // FIXME -} - -void AvatarData::setPlayerAudioOffset(float audioOffset) { - // FIXME -} - -void AvatarData::setPlayerTime(float time) { - if (!_player) { - qWarning() << "No player active"; - return; - } - - _player->seek(time * MSECS_PER_SECOND); -} - -void AvatarData::setPlayFromCurrentLocation(bool playFromCurrentLocation) { - // FIXME -} - -void AvatarData::setPlayerLoop(bool loop) { - if (_player) { - _player->loop(loop); - } -} - -void AvatarData::setPlayerUseDisplayName(bool useDisplayName) { - // FIXME -} - -void AvatarData::setPlayerUseAttachments(bool useAttachments) { - // FIXME -} - -void AvatarData::setPlayerUseHeadModel(bool useHeadModel) { - // FIXME -} - -void AvatarData::setPlayerUseSkeletonModel(bool useSkeletonModel) { - // FIXME -} - -void AvatarData::play() { - if (isPlaying()) { - if (QThread::currentThread() != thread()) { - QMetaObject::invokeMethod(this, "play", Qt::BlockingQueuedConnection); - return; - } - - _player->play(); - } -} - std::shared_ptr AvatarData::getRecordingBasis() const { return _recordingBasis; } -void AvatarData::pausePlayer() { - if (!_player) { - return; - } - if (QThread::currentThread() != thread()) { - QMetaObject::invokeMethod(this, "pausePlayer", Qt::BlockingQueuedConnection); - return; - } - if (_player) { - _player->pause(); - } -} - -void AvatarData::stopPlaying() { - if (!_player) { - return; - } - if (QThread::currentThread() != thread()) { - QMetaObject::invokeMethod(this, "stopPlaying", Qt::BlockingQueuedConnection); - return; - } - if (_player) { - _player->stop(); - } -} - void AvatarData::changeReferential(Referential* ref) { delete _referential; _referential = ref; @@ -1568,44 +1424,40 @@ JointData jointDataFromJsonValue(const QJsonValue& json) { // This allows the application to decide whether playback should be relative to an avatar's // transform at the start of playback, or relative to the transform of the recorded // avatar -QByteArray avatarStateToFrame(const AvatarData* _avatar) { +QByteArray AvatarData::toFrame(const AvatarData& avatar) { QJsonObject root; - if (!_avatar->getFaceModelURL().isEmpty()) { - root[JSON_AVATAR_HEAD_MODEL] = _avatar->getFaceModelURL().toString(); + if (!avatar.getFaceModelURL().isEmpty()) { + root[JSON_AVATAR_HEAD_MODEL] = avatar.getFaceModelURL().toString(); } - if (!_avatar->getSkeletonModelURL().isEmpty()) { - root[JSON_AVATAR_BODY_MODEL] = _avatar->getSkeletonModelURL().toString(); + if (!avatar.getSkeletonModelURL().isEmpty()) { + root[JSON_AVATAR_BODY_MODEL] = avatar.getSkeletonModelURL().toString(); } - if (!_avatar->getDisplayName().isEmpty()) { - root[JSON_AVATAR_DISPLAY_NAME] = _avatar->getDisplayName(); + if (!avatar.getDisplayName().isEmpty()) { + root[JSON_AVATAR_DISPLAY_NAME] = avatar.getDisplayName(); } - if (!_avatar->getAttachmentData().isEmpty()) { + if (!avatar.getAttachmentData().isEmpty()) { // FIXME serialize attachment data } - auto recordingBasis = _avatar->getRecordingBasis(); + auto recordingBasis = avatar.getRecordingBasis(); if (recordingBasis) { + root[JSON_AVATAR_BASIS] = Transform::toJson(*recordingBasis); // Find the relative transform - auto relativeTransform = recordingBasis->relativeTransform(_avatar->getTransform()); - - // if the resulting relative basis is identity, we shouldn't record anything - if (!relativeTransform.isIdentity()) { - root[JSON_AVATAR_RELATIVE] = Transform::toJson(relativeTransform); - root[JSON_AVATAR_BASIS] = Transform::toJson(*recordingBasis); - } + auto relativeTransform = recordingBasis->relativeTransform(avatar.getTransform()); + root[JSON_AVATAR_RELATIVE] = Transform::toJson(relativeTransform); } else { - root[JSON_AVATAR_RELATIVE] = Transform::toJson(_avatar->getTransform()); + root[JSON_AVATAR_RELATIVE] = Transform::toJson(avatar.getTransform()); } // Skeleton pose QJsonArray jointArray; - for (const auto& joint : _avatar->getRawJointData()) { + for (const auto& joint : avatar.getRawJointData()) { jointArray.push_back(toJsonValue(joint)); } root[JSON_AVATAR_JOINT_ARRAY] = jointArray; - const HeadData* head = _avatar->getHeadData(); + const HeadData* head = avatar.getHeadData(); if (head) { QJsonObject headJson; QJsonArray blendshapeCoefficients; @@ -1616,8 +1468,8 @@ QByteArray avatarStateToFrame(const AvatarData* _avatar) { headJson[JSON_AVATAR_HEAD_ROTATION] = toJsonValue(head->getRawOrientation()); headJson[JSON_AVATAR_HEAD_LEAN_FORWARD] = QJsonValue(head->getLeanForward()); headJson[JSON_AVATAR_HEAD_LEAN_SIDEWAYS] = QJsonValue(head->getLeanSideways()); - vec3 relativeLookAt = glm::inverse(_avatar->getOrientation()) * - (head->getLookAtPosition() - _avatar->getPosition()); + vec3 relativeLookAt = glm::inverse(avatar.getOrientation()) * + (head->getLookAtPosition() - avatar.getPosition()); headJson[JSON_AVATAR_HEAD_LOOKAT] = toJsonValue(relativeLookAt); root[JSON_AVATAR_HEAD] = headJson; } @@ -1625,26 +1477,32 @@ QByteArray avatarStateToFrame(const AvatarData* _avatar) { return QJsonDocument(root).toBinaryData(); } -void avatarStateFromFrame(const QByteArray& frameData, AvatarData* _avatar) { +void AvatarData::fromFrame(const QByteArray& frameData, AvatarData& result) { QJsonDocument doc = QJsonDocument::fromBinaryData(frameData); +#ifdef WANT_JSON_DEBUG + qDebug() << doc.toJson(QJsonDocument::JsonFormat::Indented); +#endif QJsonObject root = doc.object(); if (root.contains(JSON_AVATAR_HEAD_MODEL)) { auto faceModelURL = root[JSON_AVATAR_HEAD_MODEL].toString(); - if (faceModelURL != _avatar->getFaceModelURL().toString()) { - _avatar->setFaceModelURL(faceModelURL); + if (faceModelURL != result.getFaceModelURL().toString()) { + QUrl faceModel(faceModelURL); + if (faceModel.isValid()) { + result.setFaceModelURL(faceModel); + } } } if (root.contains(JSON_AVATAR_BODY_MODEL)) { auto bodyModelURL = root[JSON_AVATAR_BODY_MODEL].toString(); - if (bodyModelURL != _avatar->getSkeletonModelURL().toString()) { - _avatar->setSkeletonModelURL(bodyModelURL); + if (bodyModelURL != result.getSkeletonModelURL().toString()) { + result.setSkeletonModelURL(bodyModelURL); } } if (root.contains(JSON_AVATAR_DISPLAY_NAME)) { auto newDisplayName = root[JSON_AVATAR_DISPLAY_NAME].toString(); - if (newDisplayName != _avatar->getDisplayName()) { - _avatar->setDisplayName(newDisplayName); + if (newDisplayName != result.getDisplayName()) { + result.setDisplayName(newDisplayName); } } @@ -1656,18 +1514,18 @@ void avatarStateFromFrame(const QByteArray& frameData, AvatarData* _avatar) { // The first is more useful for playing back recordings on your own avatar, while // the latter is more useful for playing back other avatars within your scene. - auto currentBasis = _avatar->getRecordingBasis(); + auto currentBasis = result.getRecordingBasis(); if (!currentBasis) { currentBasis = std::make_shared(Transform::fromJson(root[JSON_AVATAR_BASIS])); } auto relativeTransform = Transform::fromJson(root[JSON_AVATAR_RELATIVE]); auto worldTransform = currentBasis->worldTransform(relativeTransform); - _avatar->setPosition(worldTransform.getTranslation()); - _avatar->setOrientation(worldTransform.getRotation()); + result.setPosition(worldTransform.getTranslation()); + result.setOrientation(worldTransform.getRotation()); // TODO: find a way to record/playback the Scale of the avatar - //_avatar->setTargetScale(worldTransform.getScale().x); + //result.setTargetScale(worldTransform.getScale().x); } @@ -1689,13 +1547,13 @@ void avatarStateFromFrame(const QByteArray& frameData, AvatarData* _avatar) { for (const auto& joint : jointArray) { jointRotations.push_back(joint.rotation); } - _avatar->setJointRotations(jointRotations); + result.setJointRotations(jointRotations); } #if 0 // Most head data is relative to the avatar, and needs no basis correction, // but the lookat vector does need correction - HeadData* head = _avatar->_headData; + HeadData* head = result._headData; if (head && root.contains(JSON_AVATAR_HEAD)) { QJsonObject headJson = root[JSON_AVATAR_HEAD].toObject(); if (headJson.contains(JSON_AVATAR_HEAD_BLENDSHAPE_COEFFICIENTS)) { @@ -1718,7 +1576,7 @@ void avatarStateFromFrame(const QByteArray& frameData, AvatarData* _avatar) { if (headJson.contains(JSON_AVATAR_HEAD_LOOKAT)) { auto relativeLookAt = vec3FromJsonValue(headJson[JSON_AVATAR_HEAD_LOOKAT]); if (glm::length2(relativeLookAt) > 0.01) { - head->setLookAtPosition((_avatar->getOrientation() * relativeLookAt) + _avatar->getPosition()); + head->setLookAtPosition((result.getOrientation() * relativeLookAt) + result.getPosition()); } } } diff --git a/libraries/avatars/src/AvatarData.h b/libraries/avatars/src/AvatarData.h index 26bc9d83ff..846c314e4b 100644 --- a/libraries/avatars/src/AvatarData.h +++ b/libraries/avatars/src/AvatarData.h @@ -55,8 +55,6 @@ typedef unsigned long long quint64; #include "HandData.h" #include "HeadData.h" #include "PathUtils.h" -#include "Player.h" -#include "Recorder.h" #include "Referential.h" using AvatarSharedPointer = std::shared_ptr; @@ -165,7 +163,13 @@ class AvatarData : public QObject { Q_PROPERTY(QStringList jointNames READ getJointNames) Q_PROPERTY(QUuid sessionUUID READ getSessionUUID) + public: + static const QString FRAME_NAME; + + static void fromFrame(const QByteArray& frameData, AvatarData& avatar); + static QByteArray toFrame(const AvatarData& avatar); + AvatarData(); virtual ~AvatarData(); @@ -348,25 +352,6 @@ public slots: void setJointMappingsFromNetworkReply(); void setSessionUUID(const QUuid& sessionUUID) { _sessionUUID = sessionUUID; } bool hasReferential(); - - bool isPlaying(); - bool isPaused(); - float playerElapsed(); - float playerLength(); - void loadRecording(const QString& filename); - void startPlaying(); - void setPlayerVolume(float volume); - void setPlayerAudioOffset(float audioOffset); - void setPlayerTime(float time); - void setPlayFromCurrentLocation(bool playFromCurrentLocation); - void setPlayerLoop(bool loop); - void setPlayerUseDisplayName(bool useDisplayName); - void setPlayerUseAttachments(bool useAttachments); - void setPlayerUseHeadModel(bool useHeadModel); - void setPlayerUseSkeletonModel(bool useSkeletonModel); - void play(); - void pausePlayer(); - void stopPlaying(); protected: QUuid _sessionUUID; @@ -421,8 +406,6 @@ protected: QWeakPointer _owningAvatarMixer; - recording::DeckPointer _player; - /// Loads the joint indices, names from the FST file (if any) virtual void updateJointMappings(); void changeReferential(Referential* ref); @@ -437,7 +420,7 @@ protected: QMutex avatarLock; // Name is redundant, but it aids searches. // During recording, this holds the starting position, orientation & scale of the recorded avatar - // During playback, it holds the + // During playback, it holds the origin from which to play the relative positions in the clip TransformPointer _recordingBasis; private: diff --git a/libraries/entities/src/EntityActionInterface.h b/libraries/entities/src/EntityActionInterface.h index 01292e3840..a192661e52 100644 --- a/libraries/entities/src/EntityActionInterface.h +++ b/libraries/entities/src/EntityActionInterface.h @@ -12,11 +12,14 @@ #ifndef hifi_EntityActionInterface_h #define hifi_EntityActionInterface_h +#include #include +#include -#include "EntityItem.h" - +class EntityItem; class EntitySimulation; +using EntityItemPointer = std::shared_ptr; +using EntityItemWeakPointer = std::weak_ptr; enum EntityActionType { // keep these synchronized with actionTypeFromString and actionTypeToString @@ -34,6 +37,8 @@ public: const QUuid& getID() const { return _id; } EntityActionType getType() const { return _type; } + bool isActive() { return _active; } + virtual void removeFromSimulation(EntitySimulation* simulation) const = 0; virtual EntityItemWeakPointer getOwnerEntity() const = 0; virtual void setOwnerEntity(const EntityItemPointer ownerEntity) = 0; @@ -81,6 +86,7 @@ protected: QUuid _id; EntityActionType _type; + bool _active { false }; }; diff --git a/libraries/entities/src/EntityItem.cpp b/libraries/entities/src/EntityItem.cpp index f032dcd347..100f6dfe22 100644 --- a/libraries/entities/src/EntityItem.cpp +++ b/libraries/entities/src/EntityItem.cpp @@ -1844,3 +1844,18 @@ bool EntityItem::shouldSuppressLocationEdits() const { return false; } + +QList EntityItem::getActionsOfType(EntityActionType typeToGet) { + QList result; + + QHash::const_iterator i = _objectActions.begin(); + while (i != _objectActions.end()) { + EntityActionPointer action = i.value(); + if (action->getType() == typeToGet && action->isActive()) { + result += action; + } + i++; + } + + return result; +} diff --git a/libraries/entities/src/EntityItem.h b/libraries/entities/src/EntityItem.h index 5b47198e97..5ceccef4b1 100644 --- a/libraries/entities/src/EntityItem.h +++ b/libraries/entities/src/EntityItem.h @@ -30,6 +30,7 @@ #include "EntityTypes.h" #include "SimulationOwner.h" #include "SimulationFlags.h" +#include "EntityActionInterface.h" class EntitySimulation; class EntityTreeElement; @@ -419,7 +420,9 @@ public: void setSourceUUID(const QUuid& sourceUUID) { _sourceUUID = sourceUUID; } const QUuid& getSourceUUID() const { return _sourceUUID; } - bool matchesSourceUUID(const QUuid& sourceUUID) const { return _sourceUUID == sourceUUID; } + bool matchesSourceUUID(const QUuid& sourceUUID) const { return _sourceUUID == sourceUUID; } + + QList getActionsOfType(EntityActionType typeToGet); protected: diff --git a/libraries/entities/src/EntityTree.cpp b/libraries/entities/src/EntityTree.cpp index 027972549a..dc7c19056a 100644 --- a/libraries/entities/src/EntityTree.cpp +++ b/libraries/entities/src/EntityTree.cpp @@ -704,6 +704,14 @@ void EntityTree::fixupTerseEditLogging(EntityItemProperties& properties, QList= 0) { + QString changeHint = properties.getUserData(); + changedProperties[index] = QString("userData:") + changeHint; + } + } } int EntityTree::processEditPacketData(NLPacket& packet, const unsigned char* editData, int maxLength, diff --git a/libraries/entities/src/EntityTypes.h b/libraries/entities/src/EntityTypes.h index 30b6edbc07..3536327d18 100644 --- a/libraries/entities/src/EntityTypes.h +++ b/libraries/entities/src/EntityTypes.h @@ -20,8 +20,8 @@ #include // for RenderArgs class EntityItem; -typedef std::shared_ptr EntityItemPointer; -typedef std::weak_ptr EntityItemWeakPointer; +using EntityItemPointer = std::shared_ptr; +using EntityItemWeakPointer = std::weak_ptr; inline uint qHash(const EntityItemPointer& a, uint seed) { return qHash(a.get(), seed); diff --git a/libraries/entities/src/ParticleEffectEntityItem.cpp b/libraries/entities/src/ParticleEffectEntityItem.cpp index 263d7dce0c..06fcdb495c 100644 --- a/libraries/entities/src/ParticleEffectEntityItem.cpp +++ b/libraries/entities/src/ParticleEffectEntityItem.cpp @@ -662,7 +662,7 @@ void ParticleEffectEntityItem::stepSimulation(float deltaTime) { // move head forward _particleHeadIndex = (_particleHeadIndex + 1) % _maxParticles; } else { - float age = 1.0f - _particleLifetimes[i] / _lifespan; // 0.0 .. 1.0 + float age = _particleLifetimes[i] / _lifespan; // 0.0 .. 1.0 updateRadius(i, age); updateColor(i, age); updateAlpha(i, age); diff --git a/libraries/gl/src/gl/OffscreenQmlSurface.cpp b/libraries/gl/src/gl/OffscreenQmlSurface.cpp index b6603deb62..e8a950a16b 100644 --- a/libraries/gl/src/gl/OffscreenQmlSurface.cpp +++ b/libraries/gl/src/gl/OffscreenQmlSurface.cpp @@ -26,7 +26,7 @@ #include "OffscreenGlCanvas.h" // FIXME move to threaded rendering with Qt 5.5 -// #define QML_THREADED +//#define QML_THREADED // Time between receiving a request to render the offscreen UI actually triggering // the render. Could possibly be increased depending on the framerate we expect to @@ -72,7 +72,7 @@ public: OffscreenGlCanvas::create(shareContext); #ifdef QML_THREADED // Qt 5.5 - // _renderControl->prepareThread(_renderThread); + _renderControl->prepareThread(_renderThread); _context->moveToThread(&_thread); moveToThread(&_thread); _thread.setObjectName("QML Thread"); diff --git a/libraries/networking/src/Assignment.cpp b/libraries/networking/src/Assignment.cpp index 66c6bf2a2c..5fdedeafb9 100644 --- a/libraries/networking/src/Assignment.cpp +++ b/libraries/networking/src/Assignment.cpp @@ -30,6 +30,8 @@ Assignment::Type Assignment::typeForNodeType(NodeType_t nodeType) { return Assignment::EntityServerType; case NodeType::AssetServer: return Assignment::AssetServerType; + case NodeType::MessagesMixer: + return Assignment::MessagesMixerType; default: return Assignment::AllTypes; } @@ -131,6 +133,8 @@ const char* Assignment::getTypeName() const { return "asset-server"; case Assignment::EntityServerType: return "entity-server"; + case Assignment::MessagesMixerType: + return "messages-mixer"; default: return "unknown"; } diff --git a/libraries/networking/src/Assignment.h b/libraries/networking/src/Assignment.h index ee3d9cb5fd..9639411eec 100644 --- a/libraries/networking/src/Assignment.h +++ b/libraries/networking/src/Assignment.h @@ -30,7 +30,7 @@ public: AvatarMixerType = 1, AgentType = 2, AssetServerType = 3, - UNUSED_0 = 4, + MessagesMixerType = 4, UNUSED_1 = 5, EntityServerType = 6, AllTypes = 7 diff --git a/libraries/networking/src/DomainHandler.cpp b/libraries/networking/src/DomainHandler.cpp index df024b361d..f7d26f25c5 100644 --- a/libraries/networking/src/DomainHandler.cpp +++ b/libraries/networking/src/DomainHandler.cpp @@ -46,7 +46,13 @@ DomainHandler::DomainHandler(QObject* parent) : connect(this, &DomainHandler::completedSocketDiscovery, &_icePeer, &NetworkPeer::stopPingTimer); } -void DomainHandler::clearConnectionInfo() { +void DomainHandler::disconnect() { + // if we're currently connected to a domain, send a disconnect packet on our way out + if (_isConnected) { + sendDisconnectPacket(); + } + + // clear member variables that hold the connection state to a domain _uuid = QUuid(); _connectionToken = QUuid(); @@ -60,6 +66,18 @@ void DomainHandler::clearConnectionInfo() { setIsConnected(false); } +void DomainHandler::sendDisconnectPacket() { + // The DomainDisconnect packet is not verified - we're relying on the eventual addition of DTLS to the + // domain-server connection to stop greifing here + + // construct the disconnect packet once (an empty packet but sourced with our current session UUID) + static auto disconnectPacket = NLPacket::create(PacketType::DomainDisconnectRequest, 0); + + // send the disconnect packet to the current domain server + auto nodeList = DependencyManager::get(); + nodeList->sendUnreliablePacket(*disconnectPacket, _sockAddr); +} + void DomainHandler::clearSettings() { _settingsObject = QJsonObject(); _failedSettingsRequests = 0; @@ -67,7 +85,7 @@ void DomainHandler::clearSettings() { void DomainHandler::softReset() { qCDebug(networking) << "Resetting current domain connection information."; - clearConnectionInfo(); + disconnect(); clearSettings(); } diff --git a/libraries/networking/src/DomainHandler.h b/libraries/networking/src/DomainHandler.h index 9dd4254c30..49bab6dc28 100644 --- a/libraries/networking/src/DomainHandler.h +++ b/libraries/networking/src/DomainHandler.h @@ -35,7 +35,7 @@ class DomainHandler : public QObject { public: DomainHandler(QObject* parent = 0); - void clearConnectionInfo(); + void disconnect(); void clearSettings(); const QUuid& getUUID() const { return _uuid; } @@ -113,6 +113,7 @@ signals: void settingsReceiveFail(); private: + void sendDisconnectPacket(); void hardReset(); QUuid _uuid; diff --git a/libraries/networking/src/LimitedNodeList.cpp b/libraries/networking/src/LimitedNodeList.cpp index 75d42f55cb..4a7844ecc7 100644 --- a/libraries/networking/src/LimitedNodeList.cpp +++ b/libraries/networking/src/LimitedNodeList.cpp @@ -299,14 +299,17 @@ qint64 LimitedNodeList::sendUnreliablePacket(const NLPacket& packet, const HifiS qint64 LimitedNodeList::sendPacket(std::unique_ptr packet, const Node& destinationNode) { Q_ASSERT(!packet->isPartOfMessage()); - if (!destinationNode.getActiveSocket()) { + auto activeSocket = destinationNode.getActiveSocket(); + + if (activeSocket) { + emit dataSent(destinationNode.getType(), packet->getDataSize()); + destinationNode.recordBytesSent(packet->getDataSize()); + + return sendPacket(std::move(packet), *activeSocket, destinationNode.getConnectionSecret()); + } else { + qDebug() << "LimitedNodeList::sendPacket called without active socket for node" << destinationNode << "- not sending"; return 0; } - - emit dataSent(destinationNode.getType(), packet->getDataSize()); - destinationNode.recordBytesSent(packet->getDataSize()); - - return sendPacket(std::move(packet), *destinationNode.getActiveSocket(), destinationNode.getConnectionSecret()); } qint64 LimitedNodeList::sendPacket(std::unique_ptr packet, const HifiSockAddr& sockAddr, @@ -327,21 +330,25 @@ qint64 LimitedNodeList::sendPacket(std::unique_ptr packet, const HifiS qint64 LimitedNodeList::sendPacketList(NLPacketList& packetList, const Node& destinationNode) { auto activeSocket = destinationNode.getActiveSocket(); - if (!activeSocket) { + + if (activeSocket) { + qint64 bytesSent = 0; + auto connectionSecret = destinationNode.getConnectionSecret(); + + // close the last packet in the list + packetList.closeCurrentPacket(); + + while (!packetList._packets.empty()) { + bytesSent += sendPacket(packetList.takeFront(), *activeSocket, connectionSecret); + } + + emit dataSent(destinationNode.getType(), bytesSent); + return bytesSent; + } else { + qDebug() << "LimitedNodeList::sendPacketList called without active socket for node" << destinationNode + << " - not sending."; return 0; } - qint64 bytesSent = 0; - auto connectionSecret = destinationNode.getConnectionSecret(); - - // close the last packet in the list - packetList.closeCurrentPacket(); - - while (!packetList._packets.empty()) { - bytesSent += sendPacket(packetList.takeFront(), *activeSocket, connectionSecret); - } - - emit dataSent(destinationNode.getType(), bytesSent); - return bytesSent; } qint64 LimitedNodeList::sendPacketList(NLPacketList& packetList, const HifiSockAddr& sockAddr, @@ -372,23 +379,35 @@ qint64 LimitedNodeList::sendPacketList(std::unique_ptr packetList, } qint64 LimitedNodeList::sendPacketList(std::unique_ptr packetList, const Node& destinationNode) { - // close the last packet in the list - packetList->closeCurrentPacket(); - - for (std::unique_ptr& packet : packetList->_packets) { - NLPacket* nlPacket = static_cast(packet.get()); - collectPacketStats(*nlPacket); - fillPacketHeader(*nlPacket, destinationNode.getConnectionSecret()); + auto activeSocket = destinationNode.getActiveSocket(); + if (activeSocket) { + // close the last packet in the list + packetList->closeCurrentPacket(); + + for (std::unique_ptr& packet : packetList->_packets) { + NLPacket* nlPacket = static_cast(packet.get()); + collectPacketStats(*nlPacket); + fillPacketHeader(*nlPacket, destinationNode.getConnectionSecret()); + } + + return _nodeSocket.writePacketList(std::move(packetList), *activeSocket); + } else { + qCDebug(networking) << "LimitedNodeList::sendPacketList called without active socket for node. Not sending."; + return 0; } - - return _nodeSocket.writePacketList(std::move(packetList), *destinationNode.getActiveSocket()); } qint64 LimitedNodeList::sendPacket(std::unique_ptr packet, const Node& destinationNode, const HifiSockAddr& overridenSockAddr) { + if (overridenSockAddr.isNull() && !destinationNode.getActiveSocket()) { + qCDebug(networking) << "LimitedNodeList::sendPacket called without active socket for node. Not sending."; + return 0; + } + // use the node's active socket as the destination socket if there is no overriden socket address auto& destinationSockAddr = (overridenSockAddr.isNull()) ? *destinationNode.getActiveSocket() : overridenSockAddr; + return sendPacket(std::move(packet), destinationSockAddr, destinationNode.getConnectionSecret()); } @@ -441,7 +460,7 @@ void LimitedNodeList::reset() { _nodeSocket.clearConnections(); } -void LimitedNodeList::killNodeWithUUID(const QUuid& nodeUUID) { +bool LimitedNodeList::killNodeWithUUID(const QUuid& nodeUUID) { QReadLocker readLocker(&_nodeMutex); NodeHash::iterator it = _nodeHash.find(nodeUUID); @@ -456,7 +475,10 @@ void LimitedNodeList::killNodeWithUUID(const QUuid& nodeUUID) { } handleNodeKill(matchingNode); + return true; } + + return false; } void LimitedNodeList::processKillNode(NLPacket& packet) { @@ -508,11 +530,21 @@ SharedNodePointer LimitedNodeList::addOrUpdateNode(const QUuid& uuid, NodeType_t qCDebug(networking) << "Added" << *newNode; emit nodeAdded(newNodePointer); + if (newNodePointer->getActiveSocket()) { + emit nodeActivated(newNodePointer); + } else { + connect(newNodePointer.data(), &NetworkPeer::socketActivated, this, [=] { + emit nodeActivated(newNodePointer); + disconnect(newNodePointer.data(), &NetworkPeer::socketActivated, this, 0); + }); + } return newNodePointer; } } + + std::unique_ptr LimitedNodeList::constructPingPacket(PingType_t pingType) { int packetSize = sizeof(PingType_t) + sizeof(quint64); diff --git a/libraries/networking/src/LimitedNodeList.h b/libraries/networking/src/LimitedNodeList.h index 2488b0cf8c..26e648421a 100644 --- a/libraries/networking/src/LimitedNodeList.h +++ b/libraries/networking/src/LimitedNodeList.h @@ -230,7 +230,7 @@ public slots: virtual void sendSTUNRequest(); void sendPingPackets(); - void killNodeWithUUID(const QUuid& nodeUUID); + bool killNodeWithUUID(const QUuid& nodeUUID); signals: void dataSent(quint8 channelType, int bytes); @@ -239,6 +239,7 @@ signals: void uuidChanged(const QUuid& ownerUUID, const QUuid& oldUUID); void nodeAdded(SharedNodePointer); void nodeKilled(SharedNodePointer); + void nodeActivated(SharedNodePointer); void localSockAddrChanged(const HifiSockAddr& localSockAddr); void publicSockAddrChanged(const HifiSockAddr& publicSockAddr); diff --git a/libraries/networking/src/MessagesClient.cpp b/libraries/networking/src/MessagesClient.cpp new file mode 100644 index 0000000000..2e8b3bbb09 --- /dev/null +++ b/libraries/networking/src/MessagesClient.cpp @@ -0,0 +1,108 @@ +// +// MessagesClient.cpp +// libraries/networking/src +// +// Created by Brad hefta-Gaub on 11/16/2015. +// Copyright 2015 High Fidelity, Inc. +// +// Distributed under the Apache License, Version 2.0. +// See the accompanying file LICENSE or http://www.apache.org/licenses/LICENSE-2.0.html +// + +#include "MessagesClient.h" + +#include + +#include +#include + +#include "NetworkLogging.h" +#include "NodeList.h" +#include "PacketReceiver.h" + +MessagesClient::MessagesClient() { + setCustomDeleter([](Dependency* dependency){ + static_cast(dependency)->deleteLater(); + }); + auto nodeList = DependencyManager::get(); + auto& packetReceiver = nodeList->getPacketReceiver(); + packetReceiver.registerMessageListener(PacketType::MessagesData, this, "handleMessagesPacket"); + connect(nodeList.data(), &LimitedNodeList::nodeActivated, this, &MessagesClient::handleNodeActivated); +} + +void MessagesClient::init() { + if (QThread::currentThread() != thread()) { + QMetaObject::invokeMethod(this, "init", Qt::BlockingQueuedConnection); + } +} + +void MessagesClient::handleMessagesPacket(QSharedPointer packetList, SharedNodePointer senderNode) { + QByteArray packetData = packetList->getMessage(); + QBuffer packet{ &packetData }; + packet.open(QIODevice::ReadOnly); + + quint16 channelLength; + packet.read(reinterpret_cast(&channelLength), sizeof(channelLength)); + auto channelData = packet.read(channelLength); + QString channel = QString::fromUtf8(channelData); + + quint16 messageLength; + packet.read(reinterpret_cast(&messageLength), sizeof(messageLength)); + auto messageData = packet.read(messageLength); + QString message = QString::fromUtf8(messageData); + + emit messageReceived(channel, message, senderNode->getUUID()); +} + +void MessagesClient::sendMessage(const QString& channel, const QString& message) { + auto nodeList = DependencyManager::get(); + SharedNodePointer messagesMixer = nodeList->soloNodeOfType(NodeType::MessagesMixer); + + if (messagesMixer) { + auto packetList = NLPacketList::create(PacketType::MessagesData, QByteArray(), true, true); + + auto channelUtf8 = channel.toUtf8(); + quint16 channelLength = channelUtf8.length(); + packetList->writePrimitive(channelLength); + packetList->write(channelUtf8); + + auto messageUtf8 = message.toUtf8(); + quint16 messageLength = messageUtf8.length(); + packetList->writePrimitive(messageLength); + packetList->write(messageUtf8); + + nodeList->sendPacketList(std::move(packetList), *messagesMixer); + } +} + +void MessagesClient::subscribe(const QString& channel) { + _subscribedChannels << channel; + auto nodeList = DependencyManager::get(); + SharedNodePointer messagesMixer = nodeList->soloNodeOfType(NodeType::MessagesMixer); + + if (messagesMixer) { + auto packetList = NLPacketList::create(PacketType::MessagesSubscribe, QByteArray(), true, true); + packetList->write(channel.toUtf8()); + nodeList->sendPacketList(std::move(packetList), *messagesMixer); + } +} + +void MessagesClient::unsubscribe(const QString& channel) { + _subscribedChannels.remove(channel); + auto nodeList = DependencyManager::get(); + SharedNodePointer messagesMixer = nodeList->soloNodeOfType(NodeType::MessagesMixer); + + if (messagesMixer) { + auto packetList = NLPacketList::create(PacketType::MessagesUnsubscribe, QByteArray(), true, true); + packetList->write(channel.toUtf8()); + nodeList->sendPacketList(std::move(packetList), *messagesMixer); + } +} + +void MessagesClient::handleNodeActivated(SharedNodePointer node) { + if (node->getType() == NodeType::MessagesMixer) { + for (const auto& channel : _subscribedChannels) { + subscribe(channel); + } + } +} \ No newline at end of file diff --git a/libraries/networking/src/MessagesClient.h b/libraries/networking/src/MessagesClient.h new file mode 100644 index 0000000000..a1ae4cb5ba --- /dev/null +++ b/libraries/networking/src/MessagesClient.h @@ -0,0 +1,46 @@ +// +// MessagesClient.h +// libraries/networking/src +// +// Created by Brad hefta-Gaub on 11/16/2015. +// Copyright 2015 High Fidelity, Inc. +// +// Distributed under the Apache License, Version 2.0. +// See the accompanying file LICENSE or http://www.apache.org/licenses/LICENSE-2.0.html +// + + +#ifndef hifi_MessagesClient_h +#define hifi_MessagesClient_h + +#include + +#include + +#include "LimitedNodeList.h" +#include "NLPacket.h" +#include "Node.h" + +class MessagesClient : public QObject, public Dependency { + Q_OBJECT +public: + MessagesClient(); + + Q_INVOKABLE void init(); + + Q_INVOKABLE void sendMessage(const QString& channel, const QString& message); + Q_INVOKABLE void subscribe(const QString& channel); + Q_INVOKABLE void unsubscribe(const QString& channel); + +signals: + void messageReceived(const QString& channel, const QString& message, const QUuid& senderUUID); + +private slots: + void handleMessagesPacket(QSharedPointer packetList, SharedNodePointer senderNode); + void handleNodeActivated(SharedNodePointer node); + +protected: + QSet _subscribedChannels; +}; + +#endif diff --git a/libraries/networking/src/NetworkLogging.cpp b/libraries/networking/src/NetworkLogging.cpp index 28b209960e..01abd6ae19 100644 --- a/libraries/networking/src/NetworkLogging.cpp +++ b/libraries/networking/src/NetworkLogging.cpp @@ -13,3 +13,4 @@ Q_LOGGING_CATEGORY(networking, "hifi.networking") Q_LOGGING_CATEGORY(asset_client, "hifi.networking.asset_client") +Q_LOGGING_CATEGORY(messages_client, "hifi.networking.messages_client") diff --git a/libraries/networking/src/NetworkLogging.h b/libraries/networking/src/NetworkLogging.h index 838bbb57d2..37ebc1933d 100644 --- a/libraries/networking/src/NetworkLogging.h +++ b/libraries/networking/src/NetworkLogging.h @@ -16,5 +16,6 @@ Q_DECLARE_LOGGING_CATEGORY(networking) Q_DECLARE_LOGGING_CATEGORY(asset_client) +Q_DECLARE_LOGGING_CATEGORY(messages_client) #endif // hifi_NetworkLogging_h diff --git a/libraries/networking/src/NetworkPeer.cpp b/libraries/networking/src/NetworkPeer.cpp index 9253243a7f..da2eced05c 100644 --- a/libraries/networking/src/NetworkPeer.cpp +++ b/libraries/networking/src/NetworkPeer.cpp @@ -113,6 +113,10 @@ void NetworkPeer::setActiveSocket(HifiSockAddr* discoveredSocket) { // we're now considered connected to this peer - reset the number of connection attemps resetConnectionAttempts(); + + if (_activeSocket) { + emit socketActivated(*_activeSocket); + } } void NetworkPeer::activateLocalSocket() { diff --git a/libraries/networking/src/NetworkPeer.h b/libraries/networking/src/NetworkPeer.h index c10d44bfa9..0011f3da76 100644 --- a/libraries/networking/src/NetworkPeer.h +++ b/libraries/networking/src/NetworkPeer.h @@ -83,6 +83,7 @@ public slots: void stopPingTimer(); signals: void pingTimerTimeout(); + void socketActivated(const HifiSockAddr& sockAddr); protected: void setActiveSocket(HifiSockAddr* discoveredSocket); diff --git a/libraries/networking/src/Node.cpp b/libraries/networking/src/Node.cpp index 5fea670dd0..243dca78e2 100644 --- a/libraries/networking/src/Node.cpp +++ b/libraries/networking/src/Node.cpp @@ -32,6 +32,7 @@ void NodeType::init() { TypeNameHash.insert(NodeType::Agent, "Agent"); TypeNameHash.insert(NodeType::AudioMixer, "Audio Mixer"); TypeNameHash.insert(NodeType::AvatarMixer, "Avatar Mixer"); + TypeNameHash.insert(NodeType::MessagesMixer, "Messages Mixer"); TypeNameHash.insert(NodeType::AssetServer, "Asset Server"); TypeNameHash.insert(NodeType::Unassigned, "Unassigned"); } diff --git a/libraries/networking/src/NodeList.cpp b/libraries/networking/src/NodeList.cpp index b262904c63..e03ac47854 100644 --- a/libraries/networking/src/NodeList.cpp +++ b/libraries/networking/src/NodeList.cpp @@ -103,6 +103,7 @@ NodeList::NodeList(char newOwnerType, unsigned short socketListenPort, unsigned packetReceiver.registerListener(PacketType::DomainServerRequireDTLS, &_domainHandler, "processDTLSRequirementPacket"); packetReceiver.registerListener(PacketType::ICEPingReply, &_domainHandler, "processICEPingReplyPacket"); packetReceiver.registerListener(PacketType::DomainServerPathResponse, this, "processDomainServerPathResponse"); + packetReceiver.registerListener(PacketType::DomainServerRemovedNode, this, "processDomainServerRemovedNode"); } qint64 NodeList::sendStats(const QJsonObject& statsObject, const HifiSockAddr& destination) { @@ -218,6 +219,10 @@ void NodeList::addSetOfNodeTypesToNodeInterestSet(const NodeSet& setOfNodeTypes) } void NodeList::sendDomainServerCheckIn() { + if (_isShuttingDown) { + qCDebug(networking) << "Refusing to send a domain-server check in while shutting down."; + } + if (_publicSockAddr.isNull()) { // we don't know our public socket and we need to send it to the domain server qCDebug(networking) << "Waiting for inital public socket from STUN. Will not send domain-server check in."; @@ -513,6 +518,13 @@ void NodeList::processDomainServerAddedNode(QSharedPointer packet) { parseNodeFromPacketStream(packetStream); } +void NodeList::processDomainServerRemovedNode(QSharedPointer packet) { + // read the UUID from the packet, remove it if it exists + QUuid nodeUUID = QUuid::fromRfc4122(packet->readWithoutCopy(NUM_BYTES_RFC4122_UUID)); + qDebug() << "Received packet from domain-server to remove node with UUID" << uuidStringWithoutCurlyBraces(nodeUUID); + killNodeWithUUID(nodeUUID); +} + void NodeList::parseNodeFromPacketStream(QDataStream& packetStream) { // setup variables to read into from QDataStream qint8 nodeType; diff --git a/libraries/networking/src/NodeList.h b/libraries/networking/src/NodeList.h index 3aae3e3dfc..5b9a4e5ae5 100644 --- a/libraries/networking/src/NodeList.h +++ b/libraries/networking/src/NodeList.h @@ -66,6 +66,8 @@ public: void setAssignmentServerSocket(const HifiSockAddr& serverSocket) { _assignmentServerSocket = serverSocket; } void sendAssignment(Assignment& assignment); + + void setIsShuttingDown(bool isShuttingDown) { _isShuttingDown = isShuttingDown; } public slots: void reset(); @@ -74,6 +76,7 @@ public slots: void processDomainServerList(QSharedPointer packet); void processDomainServerAddedNode(QSharedPointer packet); + void processDomainServerRemovedNode(QSharedPointer packet); void processDomainServerPathResponse(QSharedPointer packet); void processDomainServerConnectionTokenPacket(QSharedPointer packet); @@ -114,6 +117,7 @@ private: DomainHandler _domainHandler; int _numNoReplyDomainCheckIns; HifiSockAddr _assignmentServerSocket; + bool _isShuttingDown { false }; }; #endif // hifi_NodeList_h diff --git a/libraries/networking/src/NodeType.h b/libraries/networking/src/NodeType.h index e680f218db..d4377f4610 100644 --- a/libraries/networking/src/NodeType.h +++ b/libraries/networking/src/NodeType.h @@ -23,6 +23,7 @@ namespace NodeType { const NodeType_t AudioMixer = 'M'; const NodeType_t AvatarMixer = 'W'; const NodeType_t AssetServer = 'A'; + const NodeType_t MessagesMixer = 'm'; const NodeType_t Unassigned = 1; void init(); diff --git a/libraries/networking/src/PacketReceiver.cpp b/libraries/networking/src/PacketReceiver.cpp index 9d25724f6c..07f25fee5f 100644 --- a/libraries/networking/src/PacketReceiver.cpp +++ b/libraries/networking/src/PacketReceiver.cpp @@ -97,7 +97,7 @@ void PacketReceiver::registerDirectListenerForTypes(PacketTypeList types, bool PacketReceiver::registerMessageListener(PacketType type, QObject* listener, const char* slot) { Q_ASSERT_X(listener, "PacketReceiver::registerMessageListener", "No object to register"); Q_ASSERT_X(slot, "PacketReceiver::registerMessageListener", "No slot to register"); - + QMetaMethod matchingMethod = matchingMethodForListener(type, listener, slot); if (matchingMethod.isValid()) { @@ -110,8 +110,12 @@ bool PacketReceiver::registerMessageListener(PacketType type, QObject* listener, // add the mapping _packetListListenerMap[type] = ObjectMethodPair(QPointer(listener), matchingMethod); + + qCDebug(networking) << "Registering a packet listener for packet list type" << type; + return true; } else { + qCWarning(networking) << "FAILED to Register a packet listener for packet list type" << type; return false; } } @@ -352,7 +356,7 @@ void PacketReceiver::handleVerifiedPacketList(std::unique_ptr p } } else if (it == _packetListListenerMap.end()) { - qCWarning(networking) << "No listener found for packet type" << nlPacketList->getType(); + qCWarning(networking) << "No listener found for packet list type" << nlPacketList->getType(); // insert a dummy listener so we don't print this again _packetListListenerMap.insert(nlPacketList->getType(), { nullptr, QMetaMethod() }); diff --git a/libraries/networking/src/ThreadedAssignment.cpp b/libraries/networking/src/ThreadedAssignment.cpp index 0422c03297..6855c2eec3 100644 --- a/libraries/networking/src/ThreadedAssignment.cpp +++ b/libraries/networking/src/ThreadedAssignment.cpp @@ -33,14 +33,19 @@ void ThreadedAssignment::setFinished(bool isFinished) { if (_isFinished) { qDebug() << "ThreadedAssignment::setFinished(true) called - finishing up."; - - auto& packetReceiver = DependencyManager::get()->getPacketReceiver(); + + auto nodeList = DependencyManager::get(); + + auto& packetReceiver = nodeList->getPacketReceiver(); // we should de-register immediately for any of our packets packetReceiver.unregisterListener(this); // we should also tell the packet receiver to drop packets while we're cleaning up packetReceiver.setShouldDropPackets(true); + + // send a disconnect packet to the domain + nodeList->getDomainHandler().disconnect(); if (_domainServerTimer) { // stop the domain-server check in timer by calling deleteLater so it gets cleaned up on NL thread diff --git a/libraries/networking/src/udt/PacketHeaders.cpp b/libraries/networking/src/udt/PacketHeaders.cpp index 24034ff9b3..f5c66617a8 100644 --- a/libraries/networking/src/udt/PacketHeaders.cpp +++ b/libraries/networking/src/udt/PacketHeaders.cpp @@ -14,11 +14,13 @@ #include #include +#include const QSet NON_VERIFIED_PACKETS = QSet() << PacketType::NodeJsonStats << PacketType::EntityQuery << PacketType::OctreeDataNack << PacketType::EntityEditNack - << PacketType::DomainListRequest << PacketType::StopNode; + << PacketType::DomainListRequest << PacketType::StopNode + << PacketType::DomainDisconnectRequest; const QSet NON_SOURCED_PACKETS = QSet() << PacketType::StunResponse << PacketType::CreateAssignment << PacketType::RequestAssignment @@ -29,7 +31,8 @@ const QSet NON_SOURCED_PACKETS = QSet() << PacketType::DomainSettingsRequest << PacketType::DomainSettings << PacketType::ICEServerPeerInformation << PacketType::ICEServerQuery << PacketType::ICEServerHeartbeat << PacketType::ICEPing << PacketType::ICEPingReply - << PacketType::AssignmentClientStatus << PacketType::StopNode; + << PacketType::AssignmentClientStatus << PacketType::StopNode + << PacketType::DomainServerRemovedNode; const QSet RELIABLE_PACKETS = QSet(); @@ -46,66 +49,17 @@ PacketVersion versionForPacketType(PacketType packetType) { } } -#define PACKET_TYPE_NAME_LOOKUP(x) case x: return QString(#x); - -QString nameForPacketType(PacketType packetType) { - switch (packetType) { - PACKET_TYPE_NAME_LOOKUP(PacketType::Unknown); - PACKET_TYPE_NAME_LOOKUP(PacketType::StunResponse); - PACKET_TYPE_NAME_LOOKUP(PacketType::DomainList); - PACKET_TYPE_NAME_LOOKUP(PacketType::Ping); - PACKET_TYPE_NAME_LOOKUP(PacketType::PingReply); - PACKET_TYPE_NAME_LOOKUP(PacketType::KillAvatar); - PACKET_TYPE_NAME_LOOKUP(PacketType::AvatarData); - PACKET_TYPE_NAME_LOOKUP(PacketType::InjectAudio); - PACKET_TYPE_NAME_LOOKUP(PacketType::MixedAudio); - PACKET_TYPE_NAME_LOOKUP(PacketType::MicrophoneAudioNoEcho); - PACKET_TYPE_NAME_LOOKUP(PacketType::MicrophoneAudioWithEcho); - PACKET_TYPE_NAME_LOOKUP(PacketType::BulkAvatarData); - PACKET_TYPE_NAME_LOOKUP(PacketType::SilentAudioFrame); - PACKET_TYPE_NAME_LOOKUP(PacketType::DomainListRequest); - PACKET_TYPE_NAME_LOOKUP(PacketType::RequestAssignment); - PACKET_TYPE_NAME_LOOKUP(PacketType::CreateAssignment); - PACKET_TYPE_NAME_LOOKUP(PacketType::DomainConnectionDenied); - PACKET_TYPE_NAME_LOOKUP(PacketType::MuteEnvironment); - PACKET_TYPE_NAME_LOOKUP(PacketType::AudioStreamStats); - PACKET_TYPE_NAME_LOOKUP(PacketType::OctreeStats); - PACKET_TYPE_NAME_LOOKUP(PacketType::Jurisdiction); - PACKET_TYPE_NAME_LOOKUP(PacketType::JurisdictionRequest); - PACKET_TYPE_NAME_LOOKUP(PacketType::AvatarIdentity); - PACKET_TYPE_NAME_LOOKUP(PacketType::AvatarBillboard); - PACKET_TYPE_NAME_LOOKUP(PacketType::DomainConnectRequest); - PACKET_TYPE_NAME_LOOKUP(PacketType::DomainServerRequireDTLS); - PACKET_TYPE_NAME_LOOKUP(PacketType::NodeJsonStats); - PACKET_TYPE_NAME_LOOKUP(PacketType::EntityQuery); - PACKET_TYPE_NAME_LOOKUP(PacketType::EntityData); - PACKET_TYPE_NAME_LOOKUP(PacketType::EntityErase); - PACKET_TYPE_NAME_LOOKUP(PacketType::OctreeDataNack); - PACKET_TYPE_NAME_LOOKUP(PacketType::StopNode); - PACKET_TYPE_NAME_LOOKUP(PacketType::AudioEnvironment); - PACKET_TYPE_NAME_LOOKUP(PacketType::EntityEditNack); - PACKET_TYPE_NAME_LOOKUP(PacketType::ICEServerHeartbeat); - PACKET_TYPE_NAME_LOOKUP(PacketType::DomainServerAddedNode); - PACKET_TYPE_NAME_LOOKUP(PacketType::ICEServerQuery); - PACKET_TYPE_NAME_LOOKUP(PacketType::ICEServerPeerInformation); - PACKET_TYPE_NAME_LOOKUP(PacketType::ICEPing); - PACKET_TYPE_NAME_LOOKUP(PacketType::ICEPingReply); - PACKET_TYPE_NAME_LOOKUP(PacketType::EntityAdd); - PACKET_TYPE_NAME_LOOKUP(PacketType::EntityEdit); - PACKET_TYPE_NAME_LOOKUP(PacketType::DomainServerConnectionToken); - default: - return QString("Type: ") + QString::number((int)packetType); - } - return QString("unexpected"); -} - uint qHash(const PacketType& key, uint seed) { - // seems odd that Qt couldn't figure out this cast itself, but this fixes a compile error after switch to - // strongly typed enum for PacketType + // seems odd that Qt couldn't figure out this cast itself, but this fixes a compile error after switch + // to strongly typed enum for PacketType return qHash((quint8) key, seed); } QDebug operator<<(QDebug debug, const PacketType& type) { - debug.nospace() << (uint8_t) type << " (" << qPrintable(nameForPacketType(type)) << ")"; + QMetaObject metaObject = PacketTypeEnum::staticMetaObject; + QMetaEnum metaEnum = metaObject.enumerator(metaObject.enumeratorOffset()); + QString typeName = metaEnum.valueToKey((int) type); + + debug.nospace().noquote() << (uint8_t) type << " (" << typeName << ")"; return debug.space(); } diff --git a/libraries/networking/src/udt/PacketHeaders.h b/libraries/networking/src/udt/PacketHeaders.h index 82d905bf28..e0a847dcc6 100644 --- a/libraries/networking/src/udt/PacketHeaders.h +++ b/libraries/networking/src/udt/PacketHeaders.h @@ -18,70 +18,84 @@ #include #include +#include #include #include -// If adding a new packet packetType, you can replace one marked usable or add at the end. -// If you want the name of the packet packetType to be available for debugging or logging, update nameForPacketType() as well -// This enum must hold 256 or fewer packet types (so the value is <= 255) since it is statically typed as a uint8_t -enum class PacketType : uint8_t { - Unknown, - StunResponse, - DomainList, - Ping, - PingReply, - KillAvatar, - AvatarData, - InjectAudio, - MixedAudio, - MicrophoneAudioNoEcho, - MicrophoneAudioWithEcho, - BulkAvatarData, - SilentAudioFrame, - DomainListRequest, - RequestAssignment, - CreateAssignment, - DomainConnectionDenied, - MuteEnvironment, - AudioStreamStats, - DomainServerPathQuery, - DomainServerPathResponse, - DomainServerAddedNode, - ICEServerPeerInformation, - ICEServerQuery, - OctreeStats, - Jurisdiction, - JurisdictionRequest, - AssignmentClientStatus, - NoisyMute, - AvatarIdentity, - AvatarBillboard, - DomainConnectRequest, - DomainServerRequireDTLS, - NodeJsonStats, - OctreeDataNack, - StopNode, - AudioEnvironment, - EntityEditNack, - ICEServerHeartbeat, - ICEPing, - ICEPingReply, - EntityData, - EntityQuery, - EntityAdd, - EntityErase, - EntityEdit, - DomainServerConnectionToken, - DomainSettingsRequest, - DomainSettings, - AssetGet, - AssetGetReply, - AssetUpload, - AssetUploadReply, - AssetGetInfo, - AssetGetInfoReply +// The enums are inside this PacketTypeEnum for run-time conversion of enum value to string via +// Q_ENUMS, without requiring a macro that is called for each enum value. +class PacketTypeEnum { + Q_GADGET + Q_ENUMS(Value) +public: + // If adding a new packet packetType, you can replace one marked usable or add at the end. + // This enum must hold 256 or fewer packet types (so the value is <= 255) since it is statically typed as a uint8_t + enum class Value : uint8_t { + Unknown, + StunResponse, + DomainList, + Ping, + PingReply, + KillAvatar, + AvatarData, + InjectAudio, + MixedAudio, + MicrophoneAudioNoEcho, + MicrophoneAudioWithEcho, + BulkAvatarData, + SilentAudioFrame, + DomainListRequest, + RequestAssignment, + CreateAssignment, + DomainConnectionDenied, + MuteEnvironment, + AudioStreamStats, + DomainServerPathQuery, + DomainServerPathResponse, + DomainServerAddedNode, + ICEServerPeerInformation, + ICEServerQuery, + OctreeStats, + Jurisdiction, + JurisdictionRequest, + AssignmentClientStatus, + NoisyMute, + AvatarIdentity, + AvatarBillboard, + DomainConnectRequest, + DomainServerRequireDTLS, + NodeJsonStats, + OctreeDataNack, + StopNode, + AudioEnvironment, + EntityEditNack, + ICEServerHeartbeat, + ICEPing, + ICEPingReply, + EntityData, + EntityQuery, + EntityAdd, + EntityErase, + EntityEdit, + DomainServerConnectionToken, + DomainSettingsRequest, + DomainSettings, + AssetGet, + AssetGetReply, + AssetUpload, + AssetUploadReply, + AssetGetInfo, + AssetGetInfoReply, + DomainDisconnectRequest, + DomainServerRemovedNode, + MessagesData, + MessagesSubscribe, + MessagesUnsubscribe + }; }; +using PacketType = PacketTypeEnum::Value; + const int NUM_BYTES_MD5_HASH = 16; typedef char PacketVersion; @@ -90,7 +104,6 @@ extern const QSet NON_VERIFIED_PACKETS; extern const QSet NON_SOURCED_PACKETS; extern const QSet RELIABLE_PACKETS; -QString nameForPacketType(PacketType packetType); PacketVersion versionForPacketType(PacketType packetType); uint qHash(const PacketType& key, uint seed); diff --git a/libraries/octree/src/Octree.cpp b/libraries/octree/src/Octree.cpp index cceb3ba706..fe92fe7745 100644 --- a/libraries/octree/src/Octree.cpp +++ b/libraries/octree/src/Octree.cpp @@ -1890,8 +1890,8 @@ bool Octree::readSVOFromStream(unsigned long streamLength, QDataStream& inputStr versionForPacketType(expectedDataPacketType()), gotVersion); } } else { - qCDebug(octree) << "SVO file type mismatch. Expected: " << nameForPacketType(expectedType) - << " Got: " << nameForPacketType(gotType); + qCDebug(octree) << "SVO file type mismatch. Expected: " << expectedType + << " Got: " << gotType; } } else { diff --git a/libraries/physics/src/ObjectAction.h b/libraries/physics/src/ObjectAction.h index 45b40a9fb3..afb6745e9c 100644 --- a/libraries/physics/src/ObjectAction.h +++ b/libraries/physics/src/ObjectAction.h @@ -67,7 +67,6 @@ protected: EntityItemWeakPointer _ownerEntity; QString _tag; quint64 _expires { 0 }; // in seconds since epoch - bool _active { false }; private: int getEntityServerClockSkew() const; diff --git a/libraries/recording/src/recording/Clip.cpp b/libraries/recording/src/recording/Clip.cpp index 28e4211fe3..abe66ccb2e 100644 --- a/libraries/recording/src/recording/Clip.cpp +++ b/libraries/recording/src/recording/Clip.cpp @@ -23,7 +23,7 @@ Clip::Pointer Clip::fromFile(const QString& filePath) { return result; } -void Clip::toFile(const QString& filePath, Clip::Pointer clip) { +void Clip::toFile(const QString& filePath, const Clip::ConstPointer& clip) { FileClip::write(filePath, clip->duplicate()); } @@ -31,19 +31,10 @@ Clip::Pointer Clip::newClip() { return std::make_shared(); } -Clip::Pointer Clip::duplicate() { - Clip::Pointer result = std::make_shared(); - - Locker lock(_mutex); - Time currentPosition = position(); - seek(0); - - auto frame = nextFrame(); - while (frame) { - result->addFrame(frame); - frame = nextFrame(); - } - - seek(currentPosition); - return result; +void Clip::seek(float offset) { + seekFrameTime(Frame::secondsToFrameTime(offset)); } + +float Clip::position() const { + return Frame::frameTimeToSeconds(positionFrameTime()); +}; diff --git a/libraries/recording/src/recording/Clip.h b/libraries/recording/src/recording/Clip.h index a00ab72c98..722fadf0b2 100644 --- a/libraries/recording/src/recording/Clip.h +++ b/libraries/recording/src/recording/Clip.h @@ -16,6 +16,8 @@ #include +#include "Frame.h" + class QIODevice; namespace recording { @@ -23,16 +25,22 @@ namespace recording { class Clip { public: using Pointer = std::shared_ptr; + using ConstPointer = std::shared_ptr; virtual ~Clip() {} - Pointer duplicate(); + virtual Pointer duplicate() const = 0; - virtual Time duration() const = 0; + virtual QString getName() const = 0; + + virtual float duration() const = 0; virtual size_t frameCount() const = 0; - virtual void seek(Time offset) = 0; - virtual Time position() const = 0; + virtual void seek(float offset) final; + virtual float position() const final; + + virtual void seekFrameTime(Frame::Time offset) = 0; + virtual Frame::Time positionFrameTime() const = 0; virtual FrameConstPointer peekFrame() const = 0; virtual FrameConstPointer nextFrame() = 0; @@ -40,7 +48,7 @@ public: virtual void addFrame(FrameConstPointer) = 0; static Pointer fromFile(const QString& filePath); - static void toFile(const QString& filePath, Pointer clip); + static void toFile(const QString& filePath, const ConstPointer& clip); static Pointer newClip(); protected: diff --git a/libraries/recording/src/recording/Deck.cpp b/libraries/recording/src/recording/Deck.cpp index 10209c26d7..6f624db191 100644 --- a/libraries/recording/src/recording/Deck.cpp +++ b/libraries/recording/src/recording/Deck.cpp @@ -8,37 +8,54 @@ #include "Deck.h" +#include + #include #include #include "Clip.h" #include "Frame.h" #include "Logging.h" +#include "impl/OffsetClip.h" using namespace recording; -void Deck::queueClip(ClipPointer clip, Time timeOffset) { +Deck::Deck(QObject* parent) + : QObject(parent) {} + +void Deck::queueClip(ClipPointer clip, float timeOffset) { + Locker lock(_mutex); + if (!clip) { qCWarning(recordingLog) << "Clip invalid, ignoring"; return; } - // FIXME if the time offset is not zero, wrap the clip in a OffsetClip wrapper + // FIXME disabling multiple clips for now + _clips.clear(); + + // if the time offset is not zero, wrap in an OffsetClip + if (timeOffset != 0.0f) { + clip = std::make_shared(clip, timeOffset); + } + _clips.push_back(clip); _length = std::max(_length, clip->duration()); } void Deck::play() { + Locker lock(_mutex); if (_pause) { _pause = false; - _startEpoch = usecTimestampNow() - (_position * USECS_PER_MSEC); + _startEpoch = Frame::epochForFrameTime(_position); emit playbackStateChanged(); processFrames(); } } void Deck::pause() { + Locker lock(_mutex); if (!_pause) { _pause = true; emit playbackStateChanged(); @@ -47,9 +64,9 @@ void Deck::pause() { Clip::Pointer Deck::getNextClip() { Clip::Pointer result; - Time soonestFramePosition = INVALID_TIME; + auto soonestFramePosition = Frame::INVALID_TIME; for (const auto& clip : _clips) { - Time nextFramePosition = clip->position(); + auto nextFramePosition = clip->positionFrameTime(); if (nextFramePosition < soonestFramePosition) { result = clip; soonestFramePosition = nextFramePosition; @@ -58,11 +75,16 @@ Clip::Pointer Deck::getNextClip() { return result; } -void Deck::seek(Time position) { - _position = position; - // FIXME reset the frames to the appropriate spot +void Deck::seek(float position) { + Locker lock(_mutex); + _position = Frame::secondsToFrameTime(position); + + // Recompute the start epoch + _startEpoch = Frame::epochForFrameTime(_position); + + // reset the clips to the appropriate spot for (auto& clip : _clips) { - clip->seek(position); + clip->seekFrameTime(_position); } if (!_pause) { @@ -71,35 +93,57 @@ void Deck::seek(Time position) { } } -Time Deck::position() const { - if (_pause) { - return _position; +float Deck::position() const { + Locker lock(_mutex); + auto currentPosition = _position; + if (!_pause) { + currentPosition = Frame::frameTimeFromEpoch(_startEpoch); } - return (usecTimestampNow() - _startEpoch) / USECS_PER_MSEC; + return Frame::frameTimeToSeconds(currentPosition); } -static const Time MIN_FRAME_WAIT_INTERVAL_MS = 1; +static const Frame::Time MIN_FRAME_WAIT_INTERVAL = Frame::secondsToFrameTime(0.001f); +static const Frame::Time MAX_FRAME_PROCESSING_TIME = Frame::secondsToFrameTime(0.004f); void Deck::processFrames() { + if (qApp->thread() != QThread::currentThread()) { + qWarning() << "Processing frames must only happen on the main thread."; + return; + } + Locker lock(_mutex); if (_pause) { return; } - _position = position(); - auto triggerPosition = _position + MIN_FRAME_WAIT_INTERVAL_MS; + auto startingPosition = Frame::frameTimeFromEpoch(_startEpoch); + auto triggerPosition = startingPosition + MIN_FRAME_WAIT_INTERVAL; Clip::Pointer nextClip; + // FIXME add code to start dropping frames if we fall behind. + // Alternatively, add code to cache frames here and then process only the last frame of a given type + // ... the latter will work for Avatar, but not well for audio I suspect. + bool overLimit = false; for (nextClip = getNextClip(); nextClip; nextClip = getNextClip()) { - // If the clip is too far in the future, just break out of the handling loop - Time framePosition = nextClip->position(); - if (framePosition > triggerPosition) { + auto currentPosition = Frame::frameTimeFromEpoch(_startEpoch); + if ((currentPosition - startingPosition) >= MAX_FRAME_PROCESSING_TIME) { + qCWarning(recordingLog) << "Exceeded maximum frame processing time, breaking early"; +#ifdef WANT_RECORDING_DEBUG + qCDebug(recordingLog) << "Starting: " << currentPosition; + qCDebug(recordingLog) << "Current: " << startingPosition; + qCDebug(recordingLog) << "Trigger: " << triggerPosition; +#endif + overLimit = true; break; } + // If the clip is too far in the future, just break out of the handling loop + Frame::Time framePosition = nextClip->positionFrameTime(); + if (framePosition > triggerPosition) { + break; + } // Handle the frame and advance the clip Frame::handleFrame(nextClip->nextFrame()); } - if (!nextClip) { qCDebug(recordingLog) << "No more frames available"; // No more frames available, so handle the end of playback @@ -107,6 +151,9 @@ void Deck::processFrames() { qCDebug(recordingLog) << "Looping enabled, seeking back to beginning"; // If we have looping enabled, start the playback over seek(0); + // FIXME configure the recording scripting interface to reset the avatar basis on a loop + // if doing relative movement + emit looped(); } else { // otherwise pause playback pause(); @@ -115,9 +162,77 @@ void Deck::processFrames() { } // If we have more clip frames available, set the timer for the next one - Time nextClipPosition = nextClip->position(); - Time interval = nextClipPosition - _position; - _timer.singleShot(interval, [this] { + _position = Frame::frameTimeFromEpoch(_startEpoch); + int nextInterval = 1; + if (!overLimit) { + auto nextFrameTime = nextClip->positionFrameTime(); + nextInterval = (int)Frame::frameTimeToMilliseconds(nextFrameTime - _position); +#ifdef WANT_RECORDING_DEBUG + qCDebug(recordingLog) << "Now " << _position; + qCDebug(recordingLog) << "Next frame time " << nextInterval; +#endif + } +#ifdef WANT_RECORDING_DEBUG + qCDebug(recordingLog) << "Setting timer for next processing " << nextInterval; +#endif + _timer.singleShot(nextInterval, [this] { processFrames(); }); } + +void Deck::removeClip(const ClipConstPointer& clip) { + Locker lock(_mutex); + std::remove_if(_clips.begin(), _clips.end(), [&](const Clip::ConstPointer& testClip)->bool { + return (clip == testClip); + }); +} + +void Deck::removeClip(const QString& clipName) { + Locker lock(_mutex); + std::remove_if(_clips.begin(), _clips.end(), [&](const Clip::ConstPointer& clip)->bool { + return (clip->getName() == clipName); + }); +} + +void Deck::removeAllClips() { + Locker lock(_mutex); + _clips.clear(); +} + +Deck::ClipList Deck::getClips(const QString& clipName) const { + Locker lock(_mutex); + ClipList result = _clips; + return result; +} + + +bool Deck::isPlaying() { + Locker lock(_mutex); + return !_pause; +} + +bool Deck::isPaused() const { + Locker lock(_mutex); + return _pause; +} + +void Deck::stop() { + Locker lock(_mutex); + pause(); + seek(0.0f); +} + +float Deck::length() const { + Locker lock(_mutex); + return _length; +} + +void Deck::loop(bool enable) { + Locker lock(_mutex); + _loop = enable; +} + +bool Deck::isLooping() const { + Locker lock(_mutex); + return _loop; +} diff --git a/libraries/recording/src/recording/Deck.h b/libraries/recording/src/recording/Deck.h index 7086e9759d..1f8d58d5e1 100644 --- a/libraries/recording/src/recording/Deck.h +++ b/libraries/recording/src/recording/Deck.h @@ -12,56 +12,70 @@ #include #include +#include #include #include +#include + +#include #include "Forward.h" +#include "Frame.h" namespace recording { -class Deck : public QObject { +class Deck : public QObject, public ::Dependency { Q_OBJECT public: + using ClipList = std::list; using Pointer = std::shared_ptr; - Deck(QObject* parent = nullptr) : QObject(parent) {} + + Deck(QObject* parent = nullptr); // Place a clip on the deck for recording or playback - void queueClip(ClipPointer clip, Time timeOffset = 0.0f); + void queueClip(ClipPointer clip, float timeOffset = 0.0f); + void removeClip(const ClipConstPointer& clip); + void removeClip(const QString& clipName); + void removeAllClips(); + ClipList getClips(const QString& clipName) const; void play(); - bool isPlaying() { return !_pause; } + bool isPlaying(); void pause(); - bool isPaused() const { return _pause; } + bool isPaused() const; - void stop() { pause(); seek(0.0f); } + void stop(); - Time length() const { return _length; } + float length() const; - void loop(bool enable = true) { _loop = enable; } - bool isLooping() const { return _loop; } + void loop(bool enable = true); + bool isLooping() const; - Time position() const; - void seek(Time position); + float position() const; + void seek(float position); signals: void playbackStateChanged(); + void looped(); private: - using Clips = std::list; + using Mutex = std::recursive_mutex; + using Locker = std::unique_lock; ClipPointer getNextClip(); void processFrames(); + mutable Mutex _mutex; QTimer _timer; - Clips _clips; + ClipList _clips; quint64 _startEpoch { 0 }; - Time _position { 0 }; + Frame::Time _position { 0 }; bool _pause { true }; bool _loop { false }; - Time _length { 0 }; + float _length { 0 }; }; } diff --git a/libraries/recording/src/recording/Forward.h b/libraries/recording/src/recording/Forward.h index 4ba54e23a3..1bc9b31ea9 100644 --- a/libraries/recording/src/recording/Forward.h +++ b/libraries/recording/src/recording/Forward.h @@ -16,10 +16,6 @@ namespace recording { -using Time = uint32_t; - -static const Time INVALID_TIME = std::numeric_limits::max(); - using FrameType = uint16_t; using FrameSize = uint16_t; @@ -36,16 +32,14 @@ class Clip; using ClipPointer = std::shared_ptr; +using ClipConstPointer = std::shared_ptr; + // An interface for playing back clips class Deck; -using DeckPointer = std::shared_ptr; - // An interface for recording a single clip class Recorder; -using RecorderPointer = std::shared_ptr; - } #endif diff --git a/libraries/recording/src/recording/Frame.cpp b/libraries/recording/src/recording/Frame.cpp index 5b0116519f..bff85ea872 100644 --- a/libraries/recording/src/recording/Frame.cpp +++ b/libraries/recording/src/recording/Frame.cpp @@ -12,6 +12,9 @@ #include +#include +#include + using namespace recording; // FIXME move to shared @@ -73,7 +76,31 @@ using Locker = std::unique_lock; static Mutex mutex; static std::once_flag once; +float FrameHeader::frameTimeToSeconds(Frame::Time frameTime) { + float result = frameTime; + result /= MSECS_PER_SECOND; + return result; +} +uint32_t FrameHeader::frameTimeToMilliseconds(Frame::Time frameTime) { + return frameTime; +} + +Frame::Time FrameHeader::frameTimeFromEpoch(quint64 epoch) { + auto intervalMicros = (usecTimestampNow() - epoch); + intervalMicros /= USECS_PER_MSEC; + return (Frame::Time)(intervalMicros); +} + +quint64 FrameHeader::epochForFrameTime(Time frameTime) { + auto epoch = usecTimestampNow(); + epoch -= (frameTime * USECS_PER_MSEC); + return epoch; +} + +Frame::Time FrameHeader::secondsToFrameTime(float seconds) { + return (Time)(seconds * MSECS_PER_SECOND); +} FrameType Frame::registerFrameType(const QString& frameTypeName) { Locker lock(mutex); diff --git a/libraries/recording/src/recording/Frame.h b/libraries/recording/src/recording/Frame.h index f0f53ce144..3cc999f505 100644 --- a/libraries/recording/src/recording/Frame.h +++ b/libraries/recording/src/recording/Frame.h @@ -13,26 +13,46 @@ #include "Forward.h" #include +#include #include namespace recording { -struct Frame { +struct FrameHeader { + using Time = uint32_t; + + static const Time INVALID_TIME = UINT32_MAX; + static const FrameType TYPE_INVALID = 0xFFFF; + static const FrameType TYPE_HEADER = 0x0; + + static Time secondsToFrameTime(float seconds); + static float frameTimeToSeconds(Time frameTime); + + static uint32_t frameTimeToMilliseconds(Time frameTime); + + static Time frameTimeFromEpoch(quint64 epoch); + static quint64 epochForFrameTime(Time frameTime); + + FrameType type { TYPE_INVALID }; + Time timeOffset { 0 }; // milliseconds + + FrameHeader() {} + FrameHeader(FrameType type, Time timeOffset) + : type(type), timeOffset(timeOffset) { } +}; + +struct Frame : public FrameHeader { public: using Pointer = std::shared_ptr; using ConstPointer = std::shared_ptr; using Handler = std::function; - static const FrameType TYPE_INVALID = 0xFFFF; - static const FrameType TYPE_HEADER = 0x0; - FrameType type { TYPE_INVALID }; - Time timeOffset { 0 }; // milliseconds QByteArray data; Frame() {} Frame(FrameType type, float timeOffset, const QByteArray& data) - : type(type), timeOffset(timeOffset), data(data) {} + : FrameHeader(type, timeOffset), data(data) { } static FrameType registerFrameType(const QString& frameTypeName); static QMap getFrameTypes(); diff --git a/libraries/recording/src/recording/Recorder.cpp b/libraries/recording/src/recording/Recorder.cpp index f007367cae..aae31f8ec0 100644 --- a/libraries/recording/src/recording/Recorder.cpp +++ b/libraries/recording/src/recording/Recorder.cpp @@ -16,20 +16,23 @@ using namespace recording; -Recorder::~Recorder() { +Recorder::Recorder(QObject* parent) + : QObject(parent) {} -} - -Time Recorder::position() { +float Recorder::position() { + Locker lock(_mutex); + if (_clip) { + return _clip->duration(); + } return 0.0f; } void Recorder::start() { + Locker lock(_mutex); if (!_recording) { _recording = true; - if (!_clip) { - _clip = std::make_shared(); - } + // FIXME for now just record a new clip every time + _clip = std::make_shared(); _startEpoch = usecTimestampNow(); _timer.start(); emit recordingStateChanged(); @@ -37,6 +40,7 @@ void Recorder::start() { } void Recorder::stop() { + Locker lock(_mutex); if (_recording) { _recording = false; _elapsed = _timer.elapsed(); @@ -45,14 +49,17 @@ void Recorder::stop() { } bool Recorder::isRecording() { + Locker lock(_mutex); return _recording; } void Recorder::clear() { + Locker lock(_mutex); _clip.reset(); } void Recorder::recordFrame(FrameType type, QByteArray frameData) { + Locker lock(_mutex); if (!_recording || !_clip) { return; } @@ -65,6 +72,7 @@ void Recorder::recordFrame(FrameType type, QByteArray frameData) { } ClipPointer Recorder::getClip() { + Locker lock(_mutex); return _clip; } diff --git a/libraries/recording/src/recording/Recorder.h b/libraries/recording/src/recording/Recorder.h index f8346456d4..abbc964389 100644 --- a/libraries/recording/src/recording/Recorder.h +++ b/libraries/recording/src/recording/Recorder.h @@ -10,24 +10,25 @@ #ifndef hifi_Recording_Recorder_h #define hifi_Recording_Recorder_h -#include "Forward.h" +#include #include #include +#include + +#include "Forward.h" + namespace recording { // An interface for interacting with clips, creating them by recording or // playing them back. Also serialization to and from files / network sources -class Recorder : public QObject { +class Recorder : public QObject, public Dependency { Q_OBJECT public: - using Pointer = std::shared_ptr; + Recorder(QObject* parent = nullptr); - Recorder(QObject* parent = nullptr) : QObject(parent) {} - virtual ~Recorder(); - - Time position(); + float position(); // Start recording frames void start(); @@ -49,6 +50,10 @@ signals: void recordingStateChanged(); private: + using Mutex = std::recursive_mutex; + using Locker = std::unique_lock; + + Mutex _mutex; QElapsedTimer _timer; ClipPointer _clip; quint64 _elapsed { 0 }; diff --git a/libraries/recording/src/recording/impl/ArrayClip.h b/libraries/recording/src/recording/impl/ArrayClip.h new file mode 100644 index 0000000000..10b3580228 --- /dev/null +++ b/libraries/recording/src/recording/impl/ArrayClip.h @@ -0,0 +1,100 @@ +// +// Created by Bradley Austin Davis 2015/11/05 +// Copyright 2015 High Fidelity, Inc. +// +// Distributed under the Apache License, Version 2.0. +// See the accompanying file LICENSE or http://www.apache.org/licenses/LICENSE-2.0.html +// + +#pragma once +#ifndef hifi_Recording_Impl_ArrayClip_h +#define hifi_Recording_Impl_ArrayClip_h + +#include "../Clip.h" + +#include + +namespace recording { + +template +class ArrayClip : public Clip { +public: + virtual float duration() const override { + Locker lock(_mutex); + if (_frames.empty()) { + return 0; + } + return Frame::frameTimeToSeconds((*_frames.rbegin()).timeOffset); + } + + virtual size_t frameCount() const override { + Locker lock(_mutex); + return _frames.size(); + } + + Clip::Pointer duplicate() const { + auto result = newClip(); + Locker lock(_mutex); + for (size_t i = 0; i < _frames.size(); ++i) { + result->addFrame(readFrame(i)); + } + return result; + } + + virtual void seekFrameTime(Frame::Time offset) { + Locker lock(_mutex); + auto itr = std::lower_bound(_frames.begin(), _frames.end(), offset, + [](const T& a, Frame::Time b)->bool { + return a.timeOffset < b; + } + ); + _frameIndex = itr - _frames.begin(); + } + + virtual Frame::Time positionFrameTime() const override { + Locker lock(_mutex); + Frame::Time result = Frame::INVALID_TIME; + if (_frameIndex < _frames.size()) { + result = _frames[_frameIndex].timeOffset; + } + return result; + } + + virtual FrameConstPointer peekFrame() const override { + Locker lock(_mutex); + FrameConstPointer result; + if (_frameIndex < _frames.size()) { + result = readFrame(_frameIndex); + } + return result; + } + + virtual FrameConstPointer nextFrame() override { + Locker lock(_mutex); + FrameConstPointer result; + if (_frameIndex < _frames.size()) { + result = readFrame(_frameIndex++); + } + return result; + } + + virtual void skipFrame() override { + Locker lock(_mutex); + if (_frameIndex < _frames.size()) { + ++_frameIndex; + } + } + +protected: + virtual void reset() override { + _frameIndex = 0; + } + + virtual FrameConstPointer readFrame(size_t index) const = 0; + std::vector _frames; + mutable size_t _frameIndex { 0 }; +}; + +} + +#endif diff --git a/libraries/recording/src/recording/impl/BufferClip.cpp b/libraries/recording/src/recording/impl/BufferClip.cpp index 87bbfbfef7..c40d9dd42a 100644 --- a/libraries/recording/src/recording/impl/BufferClip.cpp +++ b/libraries/recording/src/recording/impl/BufferClip.cpp @@ -8,85 +8,40 @@ #include "BufferClip.h" -#include +#include +#include #include "../Frame.h" using namespace recording; - -void BufferClip::seek(Time offset) { - Locker lock(_mutex); - auto itr = std::lower_bound(_frames.begin(), _frames.end(), offset, - [](Frame::ConstPointer a, Time b)->bool { - return a->timeOffset < b; - } - ); - _frameIndex = itr - _frames.begin(); +QString BufferClip::getName() const { + return _name; } -Time BufferClip::position() const { - Locker lock(_mutex); - Time result = INVALID_TIME; - if (_frameIndex < _frames.size()) { - result = _frames[_frameIndex]->timeOffset; - } - return result; -} - -FrameConstPointer BufferClip::peekFrame() const { - Locker lock(_mutex); - FrameConstPointer result; - if (_frameIndex < _frames.size()) { - result = _frames[_frameIndex]; - } - return result; -} - -FrameConstPointer BufferClip::nextFrame() { - Locker lock(_mutex); - FrameConstPointer result; - if (_frameIndex < _frames.size()) { - result = _frames[_frameIndex]; - ++_frameIndex; - } - return result; -} void BufferClip::addFrame(FrameConstPointer newFrame) { if (newFrame->timeOffset < 0.0f) { throw std::runtime_error("Frames may not have negative time offsets"); } - auto currentPosition = position(); - seek(newFrame->timeOffset); - { - Locker lock(_mutex); - _frames.insert(_frames.begin() + _frameIndex, newFrame); - } - seek(currentPosition); -} - -void BufferClip::skipFrame() { Locker lock(_mutex); - if (_frameIndex < _frames.size()) { - ++_frameIndex; + auto itr = std::lower_bound(_frames.begin(), _frames.end(), newFrame->timeOffset, + [](const Frame& a, Frame::Time b)->bool { + return a.timeOffset < b; + } + ); + + auto newFrameIndex = itr - _frames.begin(); + //qDebug() << "Adding frame with time offset " << newFrame->timeOffset << " @ index " << newFrameIndex; + _frames.insert(_frames.begin() + newFrameIndex, Frame(*newFrame)); +} + +// Internal only function, needs no locking +FrameConstPointer BufferClip::readFrame(size_t frameIndex) const { + FramePointer result; + if (frameIndex < _frames.size()) { + result = std::make_shared(_frames[frameIndex]); } + return result; } - -void BufferClip::reset() { - Locker lock(_mutex); - _frameIndex = 0; -} - -Time BufferClip::duration() const { - if (_frames.empty()) { - return 0; - } - return (*_frames.rbegin())->timeOffset; -} - -size_t BufferClip::frameCount() const { - return _frames.size(); -} - diff --git a/libraries/recording/src/recording/impl/BufferClip.h b/libraries/recording/src/recording/impl/BufferClip.h index ce81dac730..1ea79f3df2 100644 --- a/libraries/recording/src/recording/impl/BufferClip.h +++ b/libraries/recording/src/recording/impl/BufferClip.h @@ -10,34 +10,24 @@ #ifndef hifi_Recording_Impl_BufferClip_h #define hifi_Recording_Impl_BufferClip_h -#include "../Clip.h" +#include "ArrayClip.h" -#include +#include namespace recording { -class BufferClip : public Clip { +class BufferClip : public ArrayClip { public: using Pointer = std::shared_ptr; - virtual ~BufferClip() {} - - virtual Time duration() const override; - virtual size_t frameCount() const override; - - virtual void seek(Time offset) override; - virtual Time position() const override; - - virtual FrameConstPointer peekFrame() const override; - virtual FrameConstPointer nextFrame() override; - virtual void skipFrame() override; + virtual QString getName() const override; virtual void addFrame(FrameConstPointer) override; private: - virtual void reset() override; - - std::vector _frames; - mutable size_t _frameIndex { 0 }; + virtual FrameConstPointer readFrame(size_t index) const override; + QString _name { QUuid().toString() }; + + //mutable size_t _frameIndex { 0 }; // FIXME - not in use }; } diff --git a/libraries/recording/src/recording/impl/FileClip.cpp b/libraries/recording/src/recording/impl/FileClip.cpp index b8e1eb26fa..80aaac4c87 100644 --- a/libraries/recording/src/recording/impl/FileClip.cpp +++ b/libraries/recording/src/recording/impl/FileClip.cpp @@ -18,15 +18,15 @@ #include "../Frame.h" #include "../Logging.h" +#include "BufferClip.h" using namespace recording; -static const qint64 MINIMUM_FRAME_SIZE = sizeof(FrameType) + sizeof(Time) + sizeof(FrameSize); - +static const qint64 MINIMUM_FRAME_SIZE = sizeof(FrameType) + sizeof(Frame::Time) + sizeof(FrameSize); static const QString FRAME_TYPE_MAP = QStringLiteral("frameTypes"); +static const QString FRAME_COMREPSSION_FLAG = QStringLiteral("compressed"); -using FrameHeaderList = std::list; using FrameTranslationMap = QMap; FrameTranslationMap parseTranslationMap(const QJsonDocument& doc) { @@ -49,19 +49,18 @@ FrameTranslationMap parseTranslationMap(const QJsonDocument& doc) { } -FrameHeaderList parseFrameHeaders(uchar* const start, const qint64& size) { - using FrameHeader = FileClip::FrameHeader; - FrameHeaderList results; +FileFrameHeaderList parseFrameHeaders(uchar* const start, const qint64& size) { + FileFrameHeaderList results; auto current = start; auto end = current + size; // Read all the frame headers // FIXME move to Frame::readHeader? while (end - current >= MINIMUM_FRAME_SIZE) { - FrameHeader header; + FileFrameHeader header; memcpy(&(header.type), current, sizeof(FrameType)); current += sizeof(FrameType); - memcpy(&(header.timeOffset), current, sizeof(Time)); - current += sizeof(Time); + memcpy(&(header.timeOffset), current, sizeof(Frame::Time)); + current += sizeof(Frame::Time); memcpy(&(header.size), current, sizeof(FrameSize)); current += sizeof(FrameSize); header.fileOffset = current - start; @@ -72,6 +71,11 @@ FrameHeaderList parseFrameHeaders(uchar* const start, const qint64& size) { current += header.size; results.push_back(header); } + qDebug() << "Parsed source data into " << results.size() << " frames"; + int i = 0; + for (const auto& frameHeader : results) { + qDebug() << "Frame " << i++ << " time " << frameHeader.timeOffset; + } return results; } @@ -89,7 +93,7 @@ FileClip::FileClip(const QString& fileName) : _file(fileName) { return; } - FrameHeaderList parsedFrameHeaders = parseFrameHeaders(_map, size); + auto parsedFrameHeaders = parseFrameHeaders(_map, size); // Verify that at least one frame exists and that the first frame is a header if (0 == parsedFrameHeaders.size()) { @@ -110,6 +114,11 @@ FileClip::FileClip(const QString& fileName) : _file(fileName) { _fileHeader = QJsonDocument::fromBinaryData(fileHeaderData); } + // Check for compression + { + _compressed = _fileHeader.object()[FRAME_COMREPSSION_FLAG].toBool(); + } + // Find the type enum translation map and fix up the frame headers { FrameTranslationMap translationMap = parseTranslationMap(_fileHeader); @@ -120,19 +129,25 @@ FileClip::FileClip(const QString& fileName) : _file(fileName) { qDebug() << translationMap; // Update the loaded headers with the frame data - _frameHeaders.reserve(parsedFrameHeaders.size()); + _frames.reserve(parsedFrameHeaders.size()); for (auto& frameHeader : parsedFrameHeaders) { if (!translationMap.contains(frameHeader.type)) { continue; } frameHeader.type = translationMap[frameHeader.type]; - _frameHeaders.push_back(frameHeader); + _frames.push_back(frameHeader); } } + +} + + +QString FileClip::getName() const { + return _file.fileName(); } // FIXME move to frame? -bool writeFrame(QIODevice& output, const Frame& frame) { +bool writeFrame(QIODevice& output, const Frame& frame, bool compressed = true) { if (frame.type == Frame::TYPE_INVALID) { qWarning() << "Attempting to write invalid frame"; return true; @@ -142,17 +157,24 @@ bool writeFrame(QIODevice& output, const Frame& frame) { if (written != sizeof(FrameType)) { return false; } - written = output.write((char*)&(frame.timeOffset), sizeof(Time)); - if (written != sizeof(Time)) { + //qDebug() << "Writing frame with time offset " << frame.timeOffset; + written = output.write((char*)&(frame.timeOffset), sizeof(Frame::Time)); + if (written != sizeof(Frame::Time)) { return false; } - uint16_t dataSize = frame.data.size(); + QByteArray frameData = frame.data; + if (compressed) { + frameData = qCompress(frameData); + } + + uint16_t dataSize = frameData.size(); written = output.write((char*)&dataSize, sizeof(FrameSize)); if (written != sizeof(uint16_t)) { return false; } + if (dataSize != 0) { - written = output.write(frame.data); + written = output.write(frameData); if (written != dataSize) { return false; } @@ -161,7 +183,8 @@ bool writeFrame(QIODevice& output, const Frame& frame) { } bool FileClip::write(const QString& fileName, Clip::Pointer clip) { - qCDebug(recordingLog) << "Writing clip to file " << fileName; + // FIXME need to move this to a different thread + //qCDebug(recordingLog) << "Writing clip to file " << fileName << " with " << clip->frameCount() << " frames"; if (0 == clip->frameCount()) { return false; @@ -182,10 +205,14 @@ bool FileClip::write(const QString& fileName, Clip::Pointer clip) { QJsonObject rootObject; rootObject.insert(FRAME_TYPE_MAP, frameTypeObj); + // Always mark new files as compressed + rootObject.insert(FRAME_COMREPSSION_FLAG, true); QByteArray headerFrameData = QJsonDocument(rootObject).toBinaryData(); - if (!writeFrame(outputFile, Frame({ Frame::TYPE_HEADER, 0, headerFrameData }))) { + // Never compress the header frame + if (!writeFrame(outputFile, Frame({ Frame::TYPE_HEADER, 0, headerFrameData }), false)) { return false; } + } clip->seek(0); @@ -207,73 +234,24 @@ FileClip::~FileClip() { } } -void FileClip::seek(Time offset) { - Locker lock(_mutex); - auto itr = std::lower_bound(_frameHeaders.begin(), _frameHeaders.end(), offset, - [](const FrameHeader& a, Time b)->bool { - return a.timeOffset < b; - } - ); - _frameIndex = itr - _frameHeaders.begin(); -} - -Time FileClip::position() const { - Locker lock(_mutex); - Time result = INVALID_TIME; - if (_frameIndex < _frameHeaders.size()) { - result = _frameHeaders[_frameIndex].timeOffset; - } - return result; -} - -FramePointer FileClip::readFrame(uint32_t frameIndex) const { +// Internal only function, needs no locking +FrameConstPointer FileClip::readFrame(size_t frameIndex) const { FramePointer result; - if (frameIndex < _frameHeaders.size()) { + if (frameIndex < _frames.size()) { result = std::make_shared(); - const FrameHeader& header = _frameHeaders[frameIndex]; + const auto& header = _frames[frameIndex]; result->type = header.type; result->timeOffset = header.timeOffset; if (header.size) { result->data.insert(0, reinterpret_cast(_map)+header.fileOffset, header.size); + if (_compressed) { + result->data = qUncompress(result->data); + } } } return result; } -FrameConstPointer FileClip::peekFrame() const { - Locker lock(_mutex); - return readFrame(_frameIndex); -} - -FrameConstPointer FileClip::nextFrame() { - Locker lock(_mutex); - auto result = readFrame(_frameIndex); - if (_frameIndex < _frameHeaders.size()) { - ++_frameIndex; - } - return result; -} - -void FileClip::skipFrame() { - ++_frameIndex; -} - -void FileClip::reset() { - _frameIndex = 0; -} - void FileClip::addFrame(FrameConstPointer) { throw std::runtime_error("File clips are read only"); } - -Time FileClip::duration() const { - if (_frameHeaders.empty()) { - return 0; - } - return _frameHeaders.rbegin()->timeOffset; -} - -size_t FileClip::frameCount() const { - return _frameHeaders.size(); -} - diff --git a/libraries/recording/src/recording/impl/FileClip.h b/libraries/recording/src/recording/impl/FileClip.h index 18c62936c1..f103a9aca6 100644 --- a/libraries/recording/src/recording/impl/FileClip.h +++ b/libraries/recording/src/recording/impl/FileClip.h @@ -10,31 +10,34 @@ #ifndef hifi_Recording_Impl_FileClip_h #define hifi_Recording_Impl_FileClip_h -#include "../Clip.h" +#include "ArrayClip.h" + +#include #include #include -#include +#include "../Frame.h" namespace recording { -class FileClip : public Clip { +struct FileFrameHeader : public FrameHeader { + FrameType type; + Frame::Time timeOffset; + uint16_t size; + quint64 fileOffset; +}; + +using FileFrameHeaderList = std::list; + +class FileClip : public ArrayClip { public: using Pointer = std::shared_ptr; FileClip(const QString& file); virtual ~FileClip(); - virtual Time duration() const override; - virtual size_t frameCount() const override; - - virtual void seek(Time offset) override; - virtual Time position() const override; - - virtual FrameConstPointer peekFrame() const override; - virtual FrameConstPointer nextFrame() override; - virtual void skipFrame() override; + virtual QString getName() const override; virtual void addFrame(FrameConstPointer) override; const QJsonDocument& getHeader() { @@ -43,27 +46,12 @@ public: static bool write(const QString& filePath, Clip::Pointer clip); - struct FrameHeader { - FrameType type; - Time timeOffset; - uint16_t size; - quint64 fileOffset; - }; - private: - - virtual void reset() override; - - - using FrameHeaderVector = std::vector; - - FramePointer readFrame(uint32_t frameIndex) const; - + virtual FrameConstPointer readFrame(size_t index) const override; QJsonDocument _fileHeader; QFile _file; - uint32_t _frameIndex { 0 }; uchar* _map { nullptr }; - FrameHeaderVector _frameHeaders; + bool _compressed { true }; }; } diff --git a/libraries/recording/src/recording/impl/OffsetClip.cpp b/libraries/recording/src/recording/impl/OffsetClip.cpp index bccd48d6c8..afca9e0b7a 100644 --- a/libraries/recording/src/recording/impl/OffsetClip.cpp +++ b/libraries/recording/src/recording/impl/OffsetClip.cpp @@ -22,15 +22,15 @@ using namespace recording; -OffsetClip::OffsetClip(const Clip::Pointer& wrappedClip, Time offset) - : WrapperClip(wrappedClip), _offset(offset) { } +OffsetClip::OffsetClip(const Clip::Pointer& wrappedClip, float offset) + : WrapperClip(wrappedClip), _offset(Frame::secondsToFrameTime(offset)) { } -void OffsetClip::seek(Time offset) { - _wrappedClip->seek(offset - _offset); +void OffsetClip::seekFrameTime(Frame::Time offset) { + _wrappedClip->seekFrameTime(offset - _offset); } -Time OffsetClip::position() const { - return _wrappedClip->position() + _offset; +Frame::Time OffsetClip::positionFrameTime() const { + return _wrappedClip->positionFrameTime() + _offset; } FrameConstPointer OffsetClip::peekFrame() const { @@ -45,7 +45,18 @@ FrameConstPointer OffsetClip::nextFrame() { return result; } -Time OffsetClip::duration() const { +float OffsetClip::duration() const { return _wrappedClip->duration() + _offset; } +QString OffsetClip::getName() const { + return _wrappedClip->getName(); +} + +Clip::Pointer OffsetClip::duplicate() const { + return std::make_shared( + _wrappedClip->duplicate(), Frame::frameTimeToSeconds(_offset)); +} + + + diff --git a/libraries/recording/src/recording/impl/OffsetClip.h b/libraries/recording/src/recording/impl/OffsetClip.h index 1c6b005b65..40301adf59 100644 --- a/libraries/recording/src/recording/impl/OffsetClip.h +++ b/libraries/recording/src/recording/impl/OffsetClip.h @@ -18,18 +18,20 @@ class OffsetClip : public WrapperClip { public: using Pointer = std::shared_ptr; - OffsetClip(const Clip::Pointer& wrappedClip, Time offset); - virtual ~OffsetClip(); + OffsetClip(const Clip::Pointer& wrappedClip, float offset); - virtual Time duration() const override; - virtual void seek(Time offset) override; - virtual Time position() const override; + virtual QString getName() const override; + + virtual Clip::Pointer duplicate() const override; + virtual float duration() const override; + virtual void seekFrameTime(Frame::Time offset) override; + virtual Frame::Time positionFrameTime() const override; virtual FrameConstPointer peekFrame() const override; virtual FrameConstPointer nextFrame() override; protected: - const Time _offset; + const Frame::Time _offset; }; } diff --git a/libraries/recording/src/recording/impl/WrapperClip.cpp b/libraries/recording/src/recording/impl/WrapperClip.cpp index f2bbacabf1..955dd47a5e 100644 --- a/libraries/recording/src/recording/impl/WrapperClip.cpp +++ b/libraries/recording/src/recording/impl/WrapperClip.cpp @@ -22,11 +22,11 @@ using namespace recording; WrapperClip::WrapperClip(const Clip::Pointer& wrappedClip) : _wrappedClip(wrappedClip) { } -void WrapperClip::seek(Time offset) { - _wrappedClip->seek(offset); +void WrapperClip::seekFrameTime(Frame::Time offset) { + _wrappedClip->seekFrameTime(offset); } -Time WrapperClip::position() const { +Frame::Time WrapperClip::positionFrameTime() const { return _wrappedClip->position(); } @@ -50,7 +50,7 @@ void WrapperClip::addFrame(FrameConstPointer) { throw std::runtime_error("Wrapper clips are read only"); } -Time WrapperClip::duration() const { +float WrapperClip::duration() const { return _wrappedClip->duration(); } diff --git a/libraries/recording/src/recording/impl/WrapperClip.h b/libraries/recording/src/recording/impl/WrapperClip.h index 3fe013e0ed..77a484b5f7 100644 --- a/libraries/recording/src/recording/impl/WrapperClip.h +++ b/libraries/recording/src/recording/impl/WrapperClip.h @@ -24,13 +24,12 @@ public: using Pointer = std::shared_ptr; WrapperClip(const Clip::Pointer& wrappedClip); - virtual ~WrapperClip(); - virtual Time duration() const override; + virtual float duration() const override; virtual size_t frameCount() const override; - virtual void seek(Time offset) override; - virtual Time position() const override; + virtual void seekFrameTime(Frame::Time offset) override; + virtual Frame::Time positionFrameTime() const override; virtual FrameConstPointer peekFrame() const override; virtual FrameConstPointer nextFrame() override; diff --git a/libraries/script-engine/src/ScriptEngine.cpp b/libraries/script-engine/src/ScriptEngine.cpp index 0f62bf8cd5..c17b091643 100644 --- a/libraries/script-engine/src/ScriptEngine.cpp +++ b/libraries/script-engine/src/ScriptEngine.cpp @@ -24,6 +24,7 @@ #include #include #include +#include #include #include #include @@ -366,6 +367,7 @@ void ScriptEngine::init() { registerGlobalObject("Vec3", &_vec3Library); registerGlobalObject("Uuid", &_uuidLibrary); registerGlobalObject("AnimationCache", DependencyManager::get().data()); + registerGlobalObject("Messages", DependencyManager::get().data()); qScriptRegisterMetaType(this, animVarMapToScriptValue, animVarMapFromScriptValue); qScriptRegisterMetaType(this, resultHandlerToScriptValue, resultHandlerFromScriptValue); diff --git a/libraries/shared/src/shared/JSONHelpers.cpp b/libraries/shared/src/shared/JSONHelpers.cpp index 52ece73490..c0a8820d95 100644 --- a/libraries/shared/src/shared/JSONHelpers.cpp +++ b/libraries/shared/src/shared/JSONHelpers.cpp @@ -27,7 +27,6 @@ QJsonValue glmToJson(const T& t) { template T glmFromJson(const QJsonValue& json) { - static const T DEFAULT_VALUE = T(); T result; if (json.isArray()) { QJsonArray array = json.toArray(); diff --git a/plugins/oculusLegacy/src/OculusLegacyDisplayPlugin.cpp b/plugins/oculusLegacy/src/OculusLegacyDisplayPlugin.cpp index 3898d586ad..ddf251778f 100644 --- a/plugins/oculusLegacy/src/OculusLegacyDisplayPlugin.cpp +++ b/plugins/oculusLegacy/src/OculusLegacyDisplayPlugin.cpp @@ -167,10 +167,7 @@ void OculusLegacyDisplayPlugin::activate() { } }); - #ifndef QT_NO_DEBUG - ovrBool result = - #endif - ovrHmd_ConfigureRendering(_hmd, &config.Config, distortionCaps, _eyeFovs, _eyeRenderDescs); + ovrBool result = ovrHmd_ConfigureRendering(_hmd, &config.Config, distortionCaps, _eyeFovs, _eyeRenderDescs); Q_ASSERT(result); }