Merge remote-tracking branch 'upstream/master' into 21448

BUILD.md

@@ -1,28 +1,25 @@
 ### Dependencies
 
-* [cmake](https://cmake.org/download/) ~> 3.3.2
+- [cmake](https://cmake.org/download/):  3.9
-* [Qt](https://www.qt.io/download-open-source) ~> 5.6.2
+- [Qt](https://www.qt.io/download-open-source):   5.9.1
-* [OpenSSL](https://www.openssl.org/community/binaries.html)
+- [OpenSSL](https://www.openssl.org/): Use the latest available version of OpenSSL to avoid security vulnerabilities.
-  * IMPORTANT: Use the latest available version of OpenSSL to avoid security vulnerabilities.
+- [VHACD](https://github.com/virneo/v-hacd)(clone this repository)(Optional)
-* [VHACD](https://github.com/virneo/v-hacd)(clone this repository)(Optional)
 
-#### CMake External Project Dependencies
+### CMake External Project Dependencies
 
-* [boostconfig](https://github.com/boostorg/config) ~> 1.58
+These dependencies need not be installed manually. They are automatically downloaded on the platforms where they are required.  
-* [Bullet Physics Engine](https://github.com/bulletphysics/bullet3/releases) ~> 2.83
+- [Bullet Physics Engine](https://github.com/bulletphysics/bullet3/releases):  2.83
-* [GLEW](http://glew.sourceforge.net/)
+- [GLEW](http://glew.sourceforge.net/):   1.13
-* [glm](https://glm.g-truc.net/0.9.5/index.html) ~> 0.9.5.4
+- [glm](https://glm.g-truc.net/0.9.8/index.html):  0.9.8
-* [gverb](https://github.com/highfidelity/gverb)
+- [Oculus SDK](https://developer.oculus.com/downloads/):   1.11 (Win32) / 0.5 (Mac)
-* [Oculus SDK](https://developer.oculus.com/downloads/) ~> 0.6 (Win32) / 0.5 (Mac / Linux)
+- [OpenVR](https://github.com/ValveSoftware/openvr):   1.0.6 (Win32 only)
-* [oglplus](http://oglplus.org/) ~> 0.63
+- [Polyvox](http://www.volumesoffun.com/):   0.2.1
-* [OpenVR](https://github.com/ValveSoftware/openvr) ~> 0.91 (Win32 only)
+- [QuaZip](https://sourceforge.net/projects/quazip/files/quazip/):   0.7.3
-* [Polyvox](http://www.volumesoffun.com/) ~> 0.2.1
+- [SDL2](https://www.libsdl.org/download-2.0.php):   2.0.3
-* [QuaZip](https://sourceforge.net/projects/quazip/files/quazip/) ~> 0.7.1
+- [Intel Threading Building Blocks](https://www.threadingbuildingblocks.org/):   4.3
-* [SDL2](https://www.libsdl.org/download-2.0.php) ~> 2.0.3
+- [Sixense](http://sixense.com/):   071615
-* [soxr](https://sourceforge.net/p/soxr/wiki/Home/) ~> 0.1.1
+- [zlib](http://www.zlib.net/):   1.28 (Win32 only)
-* [Intel Threading Building Blocks](https://www.threadingbuildingblocks.org/) ~> 4.3
+- nVidia Texture Tools:   2.1
-* [Sixense](http://sixense.com/) ~> 071615
-* [zlib](http://www.zlib.net/) ~> 1.28 (Win32 only)
 
 The above dependencies will be downloaded, built, linked and included automatically by CMake where we require them. The CMakeLists files that handle grabbing each of the following external dependencies can be found in the [cmake/externals folder](cmake/externals). The resulting downloads, source files and binaries will be placed in the `build/ext` folder in each of the subfolders for each external project.
 

BUILD_WIN.md

@@ -1,42 +1,45 @@
 This is a stand-alone guide for creating your first High Fidelity build for Windows 64-bit.
 
 ## Building High Fidelity
+Note: We are now using Visual Studio 2017 and Qt 5.9.1. If you are upgrading from Visual Studio 2013 and Qt 5.6.2, do a clean uninstall of those versions before going through this guide. 
 
-### Step 1. Installing Visual Studio 2013
+Note: The prerequisites will require about 10 GB of space on your drive.
 
-If you don't already have the Community or Professional edition of Visual Studio 2013, download and install [Visual Studio Community 2013](https://www.visualstudio.com/en-us/news/releasenotes/vs2013-community-vs). You do not need to install any of the optional components when going through the installer.
+### Step 1. Visual Studio 2017
 
-Note: Newer versions of Visual Studio are not yet compatible. 
+If you don’t have Community or Professional edition of Visual Studio 2017, download [Visual Studio Community 2017](https://www.visualstudio.com/downloads/). 
+
+When selecting components, check "Desktop development with C++." Also check "Windows 8.1 SDK and UCRT SDK" and "VC++ 2015.3 v140 toolset (x86,x64)" on the Summary toolbar on the right.
 
 ### Step 2. Installing CMake
 
-Download and install the [CMake 3.8.0 win64-x64 Installer](https://cmake.org/files/v3.8/cmake-3.8.0-win64-x64.msi). Make sure "Add CMake to system PATH for all users" is checked when going through the installer.
+Download and install the latest version of CMake 3.9. Download the file named  win64-x64 Installer from the [CMake Website](https://cmake.org/download/). Make sure to check "Add CMake to system PATH for all users" when prompted during installation.
 
 ### Step 3. Installing Qt
 
-Download and install the [Qt 5.6.2 for Windows 64-bit (VS 2013)](http://download.qt.io/official_releases/qt/5.6/5.6.2/qt-opensource-windows-x86-msvc2013_64-5.6.2.exe). 
+Download and install the [Qt Online Installer](https://www.qt.io/download-open-source/?hsCtaTracking=f977210e-de67-475f-a32b-65cec207fd03%7Cd62710cd-e1db-46aa-8d4d-2f1c1ffdacea). While installing, you only need to have the following components checked under Qt 5.9.1: "msvc2017 64-bit", "Qt WebEngine", and "Qt Script (Deprecated)".
 
-Keep the default components checked when going through the installer.
+Note: Installing the Sources is optional but recommended if you have room for them (~2GB). 
 
 ### Step 4. Setting Qt Environment Variable
 
 Go to `Control Panel > System > Advanced System Settings > Environment Variables > New...` (or search “Environment Variables” in Start Search).
 * Set "Variable name": `QT_CMAKE_PREFIX_PATH`
-* Set "Variable value": `%QT_DIR%\5.6\msvc2013_64\lib\cmake`
+* Set "Variable value": `C:\Qt\5.9.1\msvc2017_64\lib\cmake` 
 
 ### Step 5. Installing OpenSSL
 
-Download and install the [Win64 OpenSSL v1.0.2L Installer](https://slproweb.com/download/Win64OpenSSL-1_0_2L.exe).
+Download and install the Win64 OpenSSL v1.0.2 Installer[https://slproweb.com/products/Win32OpenSSL.html].  
 
 ### Step 6. Running CMake to Generate Build Files
 
 Run Command Prompt from Start and run the following commands:
-````
+```
 cd "%HIFI_DIR%"
 mkdir build
 cd build
-cmake .. -G "Visual Studio 12 Win64"
+cmake .. -G "Visual Studio 15 Win64"
-````
+```
     
 Where `%HIFI_DIR%` is the directory for the highfidelity repository.     
 
@@ -72,14 +75,6 @@ For any problems after Step #6, first try this:
 
 Remove `CMakeCache.txt` found in the `%HIFI_DIR%\build` directory.
 
-#### nmake cannot be found
-
-Make sure nmake.exe is located at the following path:
-
-    C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin
-    
-If not, add the directory where nmake is located to the PATH environment variable.
-
 #### Qt is throwing an error
 
-Make sure you have the correct version (5.6.2) installed and `QT_CMAKE_PREFIX_PATH` environment variable is set correctly.
+Make sure you have the correct version (5.9.1) installed and `QT_CMAKE_PREFIX_PATH` environment variable is set correctly.

CMakeLists.txt

@@ -1,4 +1,8 @@
-cmake_minimum_required(VERSION 3.2)
+if (WIN32)
+  cmake_minimum_required(VERSION 3.7)
+else()
+  cmake_minimum_required(VERSION 3.2)
+endif()
 
 if (USE_ANDROID_TOOLCHAIN)
   set(CMAKE_TOOLCHAIN_FILE "${CMAKE_CURRENT_SOURCE_DIR}/cmake/android/android.toolchain.cmake")
@@ -33,6 +37,10 @@ set(CMAKE_CXX_FLAGS_DEBUG  "${CMAKE_CXX_FLAGS_DEBUG} -DDEBUG")
 find_package( Threads )
 
 if (WIN32)
+  if (NOT "${CMAKE_SIZEOF_VOID_P}" EQUAL "8")
+    message( FATAL_ERROR "Only 64 bit builds supported." )
+  endif()
+
   add_definitions(-DNOMINMAX -D_CRT_SECURE_NO_WARNINGS)
 
   if (NOT WINDOW_SDK_PATH)
@@ -41,16 +49,13 @@ if (WIN32)
 
   # sets path for Microsoft SDKs
   # if you get build error about missing 'glu32' this path is likely wrong
-  if (MSVC10)
+  if (MSVC_VERSION GREATER_EQUAL 1910) # VS 2017
-    set(WINDOW_SDK_PATH "C:\\Program Files\\Microsoft SDKs\\Windows\\v7.1 " CACHE PATH "Windows SDK PATH")
+    set(WINDOW_SDK_PATH "C:/Program Files (x86)/Windows Kits/10/Lib/${CMAKE_VS_WINDOWS_TARGET_PLATFORM_VERSION}/x64" CACHE PATH "Windows SDK PATH")
-  elseif (MSVC12)
+  elseif (MSVC_VERSION GREATER_EQUAL 1800) # VS 2013
-    if ("${CMAKE_SIZEOF_VOID_P}" EQUAL "8")
-      set(WINDOW_SDK_FOLDER "x64")
-    else()
-      set(WINDOW_SDK_FOLDER "x86")
-    endif()
     set(WINDOW_SDK_PATH "C:\\Program Files (x86)\\Windows Kits\\8.1\\Lib\\winv6.3\\um\\${WINDOW_SDK_FOLDER}" CACHE PATH "Windows SDK PATH")
-  endif ()
+  else()
+    message( FATAL_ERROR "Visual Studio 2013 or higher required." )
+  endif()
 
   if (DEBUG_DISCOVERED_SDK_PATH)
     message(STATUS "The discovered Windows SDK path is ${WINDOW_SDK_PATH}")
@@ -103,7 +108,7 @@ else ()
 endif ()
 
 if (APPLE)
-  set(CMAKE_XCODE_ATTRIBUTE_CLANG_CXX_LANGUAGE_STANDARD "c++0x")
+  set(CMAKE_XCODE_ATTRIBUTE_CLANG_CXX_LANGUAGE_STANDARD "c++11")
   set(CMAKE_XCODE_ATTRIBUTE_CLANG_CXX_LIBRARY "libc++")
   set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} --stdlib=libc++")
 endif ()

assignment-client/CMakeLists.txt

@@ -13,7 +13,7 @@ setup_memory_debugger()
 link_hifi_libraries(
   audio avatars octree gpu model fbx entities
   networking animation recording shared script-engine embedded-webserver
-  physics plugins
+  controllers physics plugins midi
 )
 
 if (WIN32)

assignment-client/src/Agent.h

@@ -109,7 +109,7 @@ private:
     QHash<QUuid, quint16> _outgoingScriptAudioSequenceNumbers;
 
     AudioGate _audioGate;
-    bool _audioGateOpen { false };
+    bool _audioGateOpen { true };
     bool _isNoiseGateEnabled { false };
 
     CodecPluginPointer _codec;

assignment-client/src/audio/AudioMixer.cpp

@@ -127,7 +127,7 @@ void AudioMixer::queueReplicatedAudioPacket(QSharedPointer<ReceivedMessage> mess
     // construct a "fake" audio received message from the byte array and packet list information
     auto audioData = message->getMessage().mid(NUM_BYTES_RFC4122_UUID);
 
-    PacketType rewrittenType = REPLICATED_PACKET_MAPPING.key(message->getType());
+    PacketType rewrittenType = PacketTypeEnum::getReplicatedPacketMapping().key(message->getType());
 
     if (rewrittenType == PacketType::Unknown) {
         qDebug() << "Cannot unwrap replicated packet type not present in REPLICATED_PACKET_WRAPPING";

assignment-client/src/audio/AudioMixerClientData.cpp

@@ -125,11 +125,11 @@ void AudioMixerClientData::optionallyReplicatePacket(ReceivedMessage& message, c
         // now make sure it's a packet type that we want to replicate
 
         // first check if it is an original type that we should replicate
-        PacketType mirroredType = REPLICATED_PACKET_MAPPING.value(message.getType());
+        PacketType mirroredType = PacketTypeEnum::getReplicatedPacketMapping().value(message.getType());
 
         if (mirroredType == PacketType::Unknown) {
             // if it wasn't check if it is a replicated type that we should re-replicate
-            if (REPLICATED_PACKET_MAPPING.key(message.getType()) != PacketType::Unknown) {
+            if (PacketTypeEnum::getReplicatedPacketMapping().key(message.getType()) != PacketType::Unknown) {
                 mirroredType = message.getType();
             } else {
                 qDebug() << "Packet passed to optionallyReplicatePacket was not a replicatable type - returning";

assignment-client/src/avatars/AvatarMixer.cpp

@@ -144,10 +144,10 @@ void AvatarMixer::optionallyReplicatePacket(ReceivedMessage& message, const Node
         // check if this is a packet type we replicate
         // which means it must be a packet type present in REPLICATED_PACKET_MAPPING or must be the
         // replicated version of one of those packet types
-        PacketType replicatedType = REPLICATED_PACKET_MAPPING.value(message.getType());
+        PacketType replicatedType = PacketTypeEnum::getReplicatedPacketMapping().value(message.getType());
 
         if (replicatedType == PacketType::Unknown) {
-            if (REPLICATED_PACKET_MAPPING.key(message.getType()) != PacketType::Unknown) {
+            if (PacketTypeEnum::getReplicatedPacketMapping().key(message.getType()) != PacketType::Unknown) {
                 replicatedType = message.getType();
             } else {
                 qDebug() << __FUNCTION__ << "called without replicatable packet type - returning";

cmake/externals/quazip/CMakeLists.txt

@@ -21,8 +21,8 @@ endif ()
 
 ExternalProject_Add(
   ${EXTERNAL_NAME}
-  URL https://s3-us-west-1.amazonaws.com/hifi-production/dependencies/quazip-0.7.2.zip
+  URL https://hifi-public.s3.amazonaws.com/dependencies/quazip-0.7.3.zip
-  URL_MD5 2955176048a31262c09259ca8d309d19
+  URL_MD5 ed03754d39b9da1775771819b8001d45
   BINARY_DIR ${EXTERNAL_PROJECT_PREFIX}/build
   CMAKE_ARGS ${QUAZIP_CMAKE_ARGS}
   LOG_DOWNLOAD 1

cmake/externals/wasapi/CMakeLists.txt

@@ -6,8 +6,8 @@ if (WIN32)
   include(ExternalProject)
   ExternalProject_Add(
     ${EXTERNAL_NAME}
-    URL http://hifi-public.s3.amazonaws.com/dependencies/qtaudio_wasapi8.zip
+    URL http://hifi-public.s3.amazonaws.com/dependencies/qtaudio_wasapi9.zip
-    URL_MD5 b01510437ea15527156bc25cdf733bd9
+    URL_MD5 94f4765bdbcd53cd099f349ae031e769
     CONFIGURE_COMMAND ""
     BUILD_COMMAND ""
     INSTALL_COMMAND ""

cmake/macros/GenerateInstallers.cmake

@@ -22,23 +22,17 @@ macro(GENERATE_INSTALLERS)
   set(CPACK_PACKAGE_FILE_NAME "HighFidelity-Beta-${BUILD_VERSION}")
   set(CPACK_NSIS_DISPLAY_NAME ${_DISPLAY_NAME})
   set(CPACK_NSIS_PACKAGE_NAME ${_DISPLAY_NAME})
+  if (PR_BUILD)
+    set(CPACK_NSIS_COMPRESSOR "/SOLID bzip2")
+  endif ()
   set(CPACK_PACKAGE_INSTALL_DIRECTORY ${_DISPLAY_NAME})
 
+
   if (WIN32)
-    # include CMake module that will install compiler system libraries
+    # Do not install the Visual Studio C runtime libraries.  The installer will do this automatically
-    # so that we have msvcr120 and msvcp120 installed with targets
+    set(CMAKE_INSTALL_SYSTEM_RUNTIME_LIBS_SKIP TRUE)
-    set(CMAKE_INSTALL_SYSTEM_RUNTIME_DESTINATION ${INTERFACE_INSTALL_DIR})
+    
-
-    # as long as we're including sixense plugin with installer
-    # we need re-distributables for VS 2011 as well
-    # this should be removed if/when sixense support is pulled
-    set(CMAKE_INSTALL_SYSTEM_RUNTIME_LIBS
-      "${EXTERNALS_BINARY_DIR}/sixense/project/src/sixense/samples/win64/msvcr100.dll"
-      "${EXTERNALS_BINARY_DIR}/sixense/project/src/sixense/samples/win64/msvcp100.dll"
-    )
-
     include(InstallRequiredSystemLibraries)
-
     set(CPACK_NSIS_MUI_ICON "${HF_CMAKE_DIR}/installer/installer.ico")
 
     # install and reference the Add/Remove icon
@@ -90,3 +84,4 @@ macro(GENERATE_INSTALLERS)
 
   include(CPack)
 endmacro()
+

cmake/macros/InstallBesideConsole.cmake

@@ -22,9 +22,12 @@ macro(install_beside_console)
     else ()
       # setup install of executable and things copied by fixup/windeployqt
       install(
-        FILES "$<TARGET_FILE_DIR:${TARGET_NAME}>/"
+        DIRECTORY "$<TARGET_FILE_DIR:${TARGET_NAME}>/"
         DESTINATION ${COMPONENT_INSTALL_DIR}
         COMPONENT ${SERVER_COMPONENT}
+        PATTERN "*.pdb" EXCLUDE
+        PATTERN "*.lib" EXCLUDE
+        PATTERN "*.exp" EXCLUDE
       )
 
       # on windows for PR and production builds, sign the executable

cmake/macros/SetPackagingParameters.cmake

@@ -126,8 +126,14 @@ macro(SET_PACKAGING_PARAMETERS)
 
     # check if we need to find signtool
     if (PRODUCTION_BUILD OR PR_BUILD)
-      find_program(SIGNTOOL_EXECUTABLE signtool PATHS "C:/Program Files (x86)/Windows Kits/8.1" PATH_SUFFIXES "bin/x64")
+      if (MSVC_VERSION GREATER_EQUAL 1910) # VS 2017
-
+        find_program(SIGNTOOL_EXECUTABLE signtool PATHS "C:/Program Files (x86)/Windows Kits/10" PATH_SUFFIXES "bin/${CMAKE_VS_WINDOWS_TARGET_PLATFORM_VERSION}/x64")
+      elseif (MSVC_VERSION GREATER_EQUAL 1800) # VS 2013
+        find_program(SIGNTOOL_EXECUTABLE signtool PATHS "C:/Program Files (x86)/Windows Kits/8.1" PATH_SUFFIXES "bin/x64")
+      else()
+        message( FATAL_ERROR "Visual Studio 2013 or higher required." )
+      endif()
+  
       if (NOT SIGNTOOL_EXECUTABLE)
         message(FATAL_ERROR "Code signing of executables was requested but signtool.exe could not be found.")
       endif ()

cmake/templates/FixupBundlePostBuild.cmake.in

@@ -11,34 +11,28 @@
 
 include(BundleUtilities)
 
-# replace copy_resolved_item_into_bundle
+function(gp_resolved_file_type_override resolved_file type_var)
-#
+  if( file MATCHES ".*VCRUNTIME140.*" )
-# The official version of copy_resolved_item_into_bundle will print out a "warning:" when
+    set(type "system" PARENT_SCOPE)
-# the resolved item matches the resolved embedded item. This not not really an issue that
-# should rise to the level of a "warning" so we replace this message with a "status:"
-#
-function(copy_resolved_item_into_bundle resolved_item resolved_embedded_item)
-  if (WIN32)
-    # ignore case on Windows
-    string(TOLOWER "${resolved_item}" resolved_item_compare)
-    string(TOLOWER "${resolved_embedded_item}" resolved_embedded_item_compare)
-  else()
-    set(resolved_item_compare "${resolved_item}")
-    set(resolved_embedded_item_compare "${resolved_embedded_item}")
   endif()
-
+  if( file MATCHES ".*concrt140.*" )
-  if ("${resolved_item_compare}" STREQUAL "${resolved_embedded_item_compare}")
+    set(type "system" PARENT_SCOPE)
-    # this is our only change from the original version
+  endif()
-    message(STATUS "status: resolved_item == resolved_embedded_item - not copying...")
+  if( file MATCHES ".*msvcp140.*" )
-  else()
+    set(type "system" PARENT_SCOPE)
-    #message(STATUS "copying COMMAND ${CMAKE_COMMAND} -E copy ${resolved_item} ${resolved_embedded_item}")
+  endif()
-    execute_process(COMMAND ${CMAKE_COMMAND} -E copy "${resolved_item}" "${resolved_embedded_item}")
+  if( file MATCHES ".*vcruntime140.*" )
-    if(UNIX AND NOT APPLE)
+    set(type "system" PARENT_SCOPE)
-        file(RPATH_REMOVE FILE "${resolved_embedded_item}")
+  endif()
-    endif()
+  if( file MATCHES ".*api-ms-win-crt-conio.*" )
+    set(type "system" PARENT_SCOPE)
+  endif()
+  if( file MATCHES ".*api-ms-win-core-winrt.*" )
+    set(type "system" PARENT_SCOPE)
   endif()
 endfunction()
 
+
 message(STATUS "FIXUP_LIBS for fixup_bundle called for bundle ${BUNDLE_EXECUTABLE} are @FIXUP_LIBS@")
 message(STATUS "Scanning for plugins from ${BUNDLE_PLUGIN_DIR}")
 
@@ -52,3 +46,4 @@ endif()
 
 file(GLOB EXTRA_PLUGINS "${BUNDLE_PLUGIN_DIR}/*.${PLUGIN_EXTENSION}")
 fixup_bundle("${BUNDLE_EXECUTABLE}" "${EXTRA_PLUGINS}" "@FIXUP_LIBS@")
+

cmake/templates/NSIS.template.in

@@ -49,7 +49,7 @@
     Var STR_CONTAINS_VAR_3
     Var STR_CONTAINS_VAR_4
     Var STR_RETURN_VAR
-    
+        
     Function StrContains
       Exch $STR_NEEDLE
       Exch 1
@@ -343,22 +343,29 @@ SectionEnd
 ;--------------------------------
 ;Pages
   !insertmacro MUI_PAGE_WELCOME
-
+  
   !insertmacro MUI_PAGE_LICENSE "@CPACK_RESOURCE_FILE_LICENSE@"
+  
+  Page custom InstallTypesPage ReadInstallTypes
+  
+  !define MUI_PAGE_CUSTOMFUNCTION_PRE AbortFunction
   !insertmacro MUI_PAGE_DIRECTORY
-
+  
   ;Start Menu Folder Page Configuration
   !define MUI_STARTMENUPAGE_REGISTRY_ROOT "HKLM"
   !define MUI_STARTMENUPAGE_REGISTRY_KEY "Software\@CPACK_PACKAGE_VENDOR@\@CPACK_PACKAGE_INSTALL_REGISTRY_KEY@"
   !define MUI_STARTMENUPAGE_REGISTRY_VALUENAME "Start Menu Folder"
+  
+  !define MUI_PAGE_CUSTOMFUNCTION_PRE AbortFunction
   !insertmacro MUI_PAGE_STARTMENU Application $STARTMENU_FOLDER
-
+  
+  !define MUI_PAGE_CUSTOMFUNCTION_PRE AbortFunction
   @CPACK_NSIS_PAGE_COMPONENTS@
-
+    
   Page custom PostInstallOptionsPage ReadPostInstallOptions
 
   !insertmacro MUI_PAGE_INSTFILES
-
+  
   !insertmacro MUI_UNPAGE_CONFIRM
   !insertmacro MUI_UNPAGE_INSTFILES
 
@@ -442,6 +449,10 @@ Var CleanInstallCheckbox
 Var CurrentOffset
 Var OffsetUnits
 Var CopyFromProductionCheckbox
+Var ExpressInstallRadioButton
+Var CustomInstallRadioButton
+Var InstallTypeDialog
+Var Express
 
 !macro SetPostInstallOption Checkbox OptionName Default
   ; reads the value for the given post install option to the registry
@@ -459,6 +470,60 @@ Var CopyFromProductionCheckbox
   ${EndIf}
 !macroend
 
+Function InstallTypesPage
+  !insertmacro MUI_HEADER_TEXT "Choose Installation Type" "Express or Custom Install"
+  
+  nsDialogs::Create 1018
+  Pop $InstallTypeDialog
+  
+  ${If} $InstallTypeDialog == error
+    Abort
+  ${EndIf}
+  
+  StrCpy $CurrentOffset 0
+  StrCpy $OffsetUnits u
+  StrCpy $Express "0"  
+  
+  ${If} ${SectionIsSelected} ${@CLIENT_COMPONENT_NAME@}
+    ${NSD_CreateRadioButton} 30% $CurrentOffset$OffsetUnits 100% 10u "Express Install (Recommended)"; $\nInstalls High Fidelity Interface and High Fidelity Sandbox"
+    pop $ExpressInstallRadioButton
+    ${NSD_OnClick} $ExpressInstallRadioButton ChangeExpressLabel
+    IntOp $CurrentOffset $CurrentOffset + 15
+  
+    ${NSD_CreateRadiobutton} 30% $CurrentOffset$OffsetUnits 100% 10u "Custom Install (Advanced)"
+    pop $CustomInstallRadioButton
+    ${NSD_OnClick} $CustomInstallRadioButton ChangeCustomLabel 
+  ${EndIf}
+  
+  ; Express Install selected by default
+  ${NSD_Check} $ExpressInstallRadioButton
+  Call ChangeExpressLabel
+  
+  nsDialogs::Show
+FunctionEnd
+
+Function ChangeExpressLabel
+  Push $R1
+  GetDlgItem $R1 $HWNDPARENT 1
+  SendMessage $R1 ${WM_SETTEXT} 0 "STR:Install"
+  Pop $R1
+FunctionEnd
+
+Function ChangeCustomLabel
+  Push $R1
+  GetDlgItem $R1 $HWNDPARENT 1
+  SendMessage $R1 ${WM_SETTEXT} 0 "STR:Next >"
+  Pop $R1
+FunctionEnd
+
+Function AbortFunction
+  ; Check if Express is set, if so, abort the post install options page
+  Call HandleInstallTypes ; Sets Express if ExpressInstallRadioButton is checked and installs with defaults
+  StrCmp $Express "1" 0 end
+  Abort
+  end:
+FunctionEnd  
+
 Function PostInstallOptionsPage
   !insertmacro MUI_HEADER_TEXT "Setup Options" ""
 
@@ -549,9 +614,15 @@ Function PostInstallOptionsPage
     ${NSD_CreateCheckbox} 0 $CurrentOffset$OffsetUnits 100% 10u "&Copy settings and content from production install"
     Pop $CopyFromProductionCheckbox
 
-    ${NSD_SetState} $CopyFromProductionCheckbox ${BST_CHECKED}
+    ${NSD_SetState} $CopyFromProductionCheckbox ${BST_UNCHECKED}
   ${EndIf}
   
+  ; Check if Express is set, if so, abort the post install options page
+  Call HandleInstallTypes ; Sets Express if ExpressInstallRadioButton is checked and installs with defaults
+  StrCmp $Express "1" 0 end
+  Abort
+  end:
+  
   nsDialogs::Show
 FunctionEnd
 
@@ -567,6 +638,16 @@ Var LaunchServerNowState
 Var LaunchClientNowState
 Var CopyFromProductionState
 Var CleanInstallState
+Var ExpressInstallState 
+Var CustomInstallState
+
+Function ReadInstallTypes
+  ${If} ${SectionIsSelected} ${@CLIENT_COMPONENT_NAME@}
+    ; check if the user asked for express/custom install
+    ${NSD_GetState} $ExpressInstallRadioButton $ExpressInstallState
+    ${NSD_GetState} $CustomInstallRadioButton $CustomInstallState
+  ${EndIf}
+FunctionEnd
 
 Function ReadPostInstallOptions
   ${If} ${SectionIsSelected} ${@CLIENT_COMPONENT_NAME@}
@@ -603,6 +684,28 @@ Function ReadPostInstallOptions
   ${EndIf}
 FunctionEnd
 
+Function HandleInstallTypes
+  ${If} $ExpressInstallState == ${BST_CHECKED}
+    
+    StrCpy $Express "1"
+    
+    ; over ride custom checkboxes and select defaults 
+    ${NSD_SetState} $DesktopClientCheckbox ${BST_CHECKED} 
+    ${NSD_SetState} $ServerStartupCheckbox ${BST_CHECKED} 
+    ${NSD_SetState} $LaunchServerNowCheckbox ${BST_CHECKED}
+    ${NSD_SetState} $LaunchClientNowCheckbox ${BST_CHECKED}
+    
+    ${If} @PR_BUILD@ == 1
+      ${NSD_SetState} $CopyFromProductionCheckbox ${BST_UNCHECKED}
+    ${EndIf}    
+    
+    ; call ReadPostInstallOptions and HandlePostInstallOptions with defaults selected
+    Call ReadPostInstallOptions 
+    Call HandlePostInstallOptions 
+      
+  ${EndIf}
+FunctionEnd
+
 Function HandlePostInstallOptions
   ${If} ${SectionIsSelected} ${@CLIENT_COMPONENT_NAME@}
     ; check if the user asked for a desktop shortcut to High Fidelity
@@ -624,6 +727,7 @@ Function HandlePostInstallOptions
       !insertmacro WritePostInstallOption @CONSOLE_DESKTOP_SHORTCUT_REG_KEY@ NO
     ${EndIf}
 
+    
     ; check if the user asked to have Sandbox launched every startup
     ${If} $ServerStartupState == ${BST_CHECKED}
       ; in case we added a shortcut in the global context, pull that now
@@ -749,6 +853,8 @@ Section "-Core installation"
   ; Rename the incorrectly cased Raleway font
   Rename "$INSTDIR\resources\qml\styles-uit\RalewaySemibold.qml" "$INSTDIR\resources\qml\styles-uit\RalewaySemiBold.qml"
 
+  ExecWait "$INSTDIR\vcredist_x64.exe /install /q /norestart"
+
   ; Remove the Old Interface directory and vcredist_x64.exe (from installs prior to Server Console)
   RMDir /r "$INSTDIR\Interface"
   Delete "$INSTDIR\vcredist_x64.exe"

domain-server/src/DomainGatekeeper.cpp

@@ -385,7 +385,7 @@ SharedNodePointer DomainGatekeeper::processAgentConnectRequest(const NodeConnect
             // user is attempting to prove their identity to us, but we don't have enough information
             sendConnectionTokenPacket(username, nodeConnection.senderSockAddr);
             // ask for their public key right now to make sure we have it
-            requestUserPublicKey(username);
+            requestUserPublicKey(username, true);
             getGroupMemberships(username); // optimistically get started on group memberships
 #ifdef WANT_DEBUG
             qDebug() << "stalling login because we have no username-signature:" << username;
@@ -521,7 +521,10 @@ bool DomainGatekeeper::verifyUserSignature(const QString& username,
                                            const HifiSockAddr& senderSockAddr) {
     // it's possible this user can be allowed to connect, but we need to check their username signature
     auto lowerUsername = username.toLower();
-    QByteArray publicKeyArray = _userPublicKeys.value(lowerUsername);
+    KeyFlagPair publicKeyPair = _userPublicKeys.value(lowerUsername);
+
+    QByteArray publicKeyArray = publicKeyPair.first;
+    bool isOptimisticKey = publicKeyPair.second;
 
     const QUuid& connectionToken = _connectionTokenHash.value(lowerUsername);
 
@@ -555,10 +558,16 @@ bool DomainGatekeeper::verifyUserSignature(const QString& username,
                 return true;
 
             } else {
-                if (!senderSockAddr.isNull()) {
+                // we only send back a LoginError if this wasn't an "optimistic" key
-                    qDebug() << "Error decrypting username signature for " << username << "- denying connection.";
+                // (a key that we hoped would work but is probably stale)
+
+                if (!senderSockAddr.isNull() && !isOptimisticKey) {
+                    qDebug() << "Error decrypting username signature for" << username << "- denying connection.";
                     sendConnectionDeniedPacket("Error decrypting username signature.", senderSockAddr,
                         DomainHandler::ConnectionRefusedReason::LoginError);
+                } else if (!senderSockAddr.isNull()) {
+                    qDebug() << "Error decrypting username signature for" << username << "with optimisitic key -"
+                        << "re-requesting public key and delaying connection";
                 }
 
                 // free up the public key, we don't need it anymore
@@ -604,20 +613,7 @@ bool DomainGatekeeper::isWithinMaxCapacity() {
     return true;
 }
 
-
+void DomainGatekeeper::requestUserPublicKey(const QString& username, bool isOptimistic) {
-void DomainGatekeeper::preloadAllowedUserPublicKeys() {
-    QStringList allowedUsers = _server->_settingsManager.getAllNames();
-
-    if (allowedUsers.size() > 0) {
-        // in the future we may need to limit how many requests here - for now assume that lists of allowed users are not
-        // going to create > 100 requests
-        foreach(const QString& username, allowedUsers) {
-            requestUserPublicKey(username);
-        }
-    }
-}
-
-void DomainGatekeeper::requestUserPublicKey(const QString& username) {
     // don't request public keys for the standard psuedo-account-names
     if (NodePermissions::standardNames.contains(username, Qt::CaseInsensitive)) {
         return;
@@ -628,7 +624,7 @@ void DomainGatekeeper::requestUserPublicKey(const QString& username) {
         // public-key request for this username is already flight, not rerequesting
         return;
     }
-    _inFlightPublicKeyRequests += lowerUsername;
+    _inFlightPublicKeyRequests.insert(lowerUsername, isOptimistic);
 
     // even if we have a public key for them right now, request a new one in case it has just changed
     JSONCallbackParameters callbackParams;
@@ -640,7 +636,7 @@ void DomainGatekeeper::requestUserPublicKey(const QString& username) {
 
     const QString USER_PUBLIC_KEY_PATH = "api/v1/users/%1/public_key";
 
-    qDebug() << "Requesting public key for user" << username;
+    qDebug().nospace() << "Requesting " << (isOptimistic ? "optimistic " : " ") << "public key for user " << username;
 
     DependencyManager::get<AccountManager>()->sendRequest(USER_PUBLIC_KEY_PATH.arg(username),
                                               AccountManagerAuth::None,
@@ -662,16 +658,21 @@ void DomainGatekeeper::publicKeyJSONCallback(QNetworkReply& requestReply) {
     QJsonObject jsonObject = QJsonDocument::fromJson(requestReply.readAll()).object();
     QString username = extractUsernameFromPublicKeyRequest(requestReply);
 
+    bool isOptimisticKey = _inFlightPublicKeyRequests.take(username);
+
     if (jsonObject["status"].toString() == "success" && !username.isEmpty()) {
         // pull the public key as a QByteArray from this response
         const QString JSON_DATA_KEY = "data";
         const QString JSON_PUBLIC_KEY_KEY = "public_key";
 
-        _userPublicKeys[username.toLower()] =
+        qDebug().nospace() << "Extracted " << (isOptimisticKey ? "optimistic " : " ") << "public key for " << username.toLower();
-            QByteArray::fromBase64(jsonObject[JSON_DATA_KEY].toObject()[JSON_PUBLIC_KEY_KEY].toString().toUtf8());
-    }
 
-    _inFlightPublicKeyRequests.remove(username);
+        _userPublicKeys[username.toLower()] =
+            {
+                QByteArray::fromBase64(jsonObject[JSON_DATA_KEY].toObject()[JSON_PUBLIC_KEY_KEY].toString().toUtf8()),
+                isOptimisticKey
+            };
+    }
 }
 
 void DomainGatekeeper::publicKeyJSONErrorCallback(QNetworkReply& requestReply) {

domain-server/src/DomainGatekeeper.h

@@ -39,8 +39,6 @@ public:
                                 const QUuid& walletUUID, const QString& nodeVersion);
     QUuid assignmentUUIDForPendingAssignment(const QUuid& tempUUID);
     
-    void preloadAllowedUserPublicKeys();
-    
     void removeICEPeer(const QUuid& peerUUID) { _icePeers.remove(peerUUID); }
 
     static void sendProtocolMismatchConnectionDenial(const HifiSockAddr& senderSockAddr);
@@ -93,7 +91,7 @@ private:
     
     void pingPunchForConnectingPeer(const SharedNetworkPeer& peer);
     
-    void requestUserPublicKey(const QString& username);
+    void requestUserPublicKey(const QString& username, bool isOptimistic = false);
     
     DomainServer* _server;
     
@@ -102,8 +100,17 @@ private:
     QHash<QUuid, SharedNetworkPeer> _icePeers;
     
     QHash<QString, QUuid> _connectionTokenHash;
-    QHash<QString, QByteArray> _userPublicKeys;
+
-    QSet<QString> _inFlightPublicKeyRequests; // keep track of which we've already asked for
+    // the word "optimistic" below is used for keys that we request during user connection before the user has
+    // had a chance to upload a new public key
+
+    // we don't send back user signature decryption errors for those keys so that there isn't a thrasing of key re-generation
+    // and connection refusal
+
+    using KeyFlagPair = QPair<QByteArray, bool>;
+
+    QHash<QString, KeyFlagPair> _userPublicKeys; // keep track of keys and flag them as optimistic or not
+    QHash<QString, bool> _inFlightPublicKeyRequests; // keep track of keys we've asked for (and if it was optimistic)
     QSet<QString> _domainOwnerFriends; // keep track of friends of the domain owner
     QSet<QString> _inFlightGroupMembershipsRequests; // keep track of which we've already asked for
 

domain-server/src/DomainServer.cpp

@@ -160,9 +160,7 @@ DomainServer::DomainServer(int argc, char* argv[]) :
         getTemporaryName();
     }
 
-    _gatekeeper.preloadAllowedUserPublicKeys(); // so they can connect on first request
+    // send signal to DomainMetadata when descriptors changed
-
-    //send signal to DomainMetadata when descriptors changed
     _metadata = new DomainMetadata(this);
     connect(&_settingsManager, &DomainServerSettingsManager::settingsUpdated,
             _metadata, &DomainMetadata::descriptorsChanged);

interface/CMakeLists.txt

@@ -195,7 +195,7 @@ link_hifi_libraries(
   shared octree ktx gpu gl gpu-gl procedural model render
   recording fbx networking model-networking entities avatars trackers
   audio audio-client animation script-engine physics
-  render-utils entities-renderer avatars-renderer ui auto-updater
+  render-utils entities-renderer avatars-renderer ui auto-updater midi
   controllers plugins image trackers
   ui-plugins display-plugins input-plugins
   ${NON_ANDROID_LIBRARIES}
@@ -309,9 +309,12 @@ else (APPLE)
 
     # setup install of executable and things copied by fixup/windeployqt
     install(
-      FILES "$<TARGET_FILE_DIR:${TARGET_NAME}>/"
+      DIRECTORY "$<TARGET_FILE_DIR:${TARGET_NAME}>/"
       DESTINATION ${INTERFACE_INSTALL_DIR}
       COMPONENT ${CLIENT_COMPONENT}
+      PATTERN "*.pdb" EXCLUDE
+      PATTERN "*.lib" EXCLUDE
+      PATTERN "*.exp" EXCLUDE
     )
 
     set(SCRIPTS_INSTALL_DIR "${INTERFACE_INSTALL_DIR}")

interface/resources/avatar/avatar-animation.json

@@ -126,24 +126,6 @@
                                     "weightVar": "headWeight",
                                     "weight": 4.0,
                                     "flexCoefficients": [1, 0.5, 0.25, 0.2, 0.1]
-                                },
-                                {
-                                    "jointName": "LeftArm",
-                                    "positionVar": "leftArmPosition",
-                                    "rotationVar": "leftArmRotation",
-                                    "typeVar": "leftArmType",
-                                    "weightVar": "leftArmWeight",
-                                    "weight": 0.75,
-                                    "flexCoefficients": [1.0, 0.35, 0.2, 0.1, 0.05, 0.0, 0.0, 0.0]
-                                },
-                                {
-                                    "jointName": "RightArm",
-                                    "positionVar": "rightArmPosition",
-                                    "rotationVar": "rightArmRotation",
-                                    "typeVar": "rightArmType",
-                                    "weightVar": "rightArmWeight",
-                                    "weight": 0.75,
-                                    "flexCoefficients": [1.0, 0.35, 0.2, 0.1, 0.05, 0.0, 0.0, 0.0]
                                 }
                             ]
                         },

interface/resources/controllers/keyboardMouse.json

@@ -15,7 +15,7 @@
         { "comment" : "Mouse turn need to be small continuous increments",
           "from": { "makeAxis" : [
                 [ "Keyboard.MouseMoveLeft" ],
-                [ "Keyboard.MouseMoveRight" ] 
+                [ "Keyboard.MouseMoveRight" ]
             ]
           },
           "when": [ "Application.InHMD", "Application.SnapTurn", "Keyboard.RightMouseButton" ],
@@ -31,8 +31,8 @@
         { "comment" : "Touchpad turn need to be small continuous increments, but without the RMB constraint",
           "from": { "makeAxis" : [
                 [ "Keyboard.TouchpadLeft" ],
-                [ "Keyboard.TouchpadRight" ] 
+                [ "Keyboard.TouchpadRight" ]
-            ] 
+            ]
           },
           "when": [ "Application.InHMD", "Application.SnapTurn" ],
           "to": "Actions.StepYaw",

interface/resources/controllers/standard.json

@@ -109,6 +109,23 @@
 
         { "from": "Standard.Head", "to": "Actions.Head" },
         { "from": "Standard.LeftArm", "to": "Actions.LeftArm" },
-        { "from": "Standard.RightArm", "to": "Actions.RightArm" }
+        { "from": "Standard.RightArm", "to": "Actions.RightArm" },
+        
+        { "from": "Standard.TrackedObject00", "to" : "Actions.TrackedObject00" },
+        { "from": "Standard.TrackedObject01", "to" : "Actions.TrackedObject01" },
+        { "from": "Standard.TrackedObject02", "to" : "Actions.TrackedObject02" },
+        { "from": "Standard.TrackedObject03", "to" : "Actions.TrackedObject03" },
+        { "from": "Standard.TrackedObject04", "to" : "Actions.TrackedObject04" },
+        { "from": "Standard.TrackedObject05", "to" : "Actions.TrackedObject05" },
+        { "from": "Standard.TrackedObject06", "to" : "Actions.TrackedObject06" },
+        { "from": "Standard.TrackedObject07", "to" : "Actions.TrackedObject07" },
+        { "from": "Standard.TrackedObject08", "to" : "Actions.TrackedObject08" },
+        { "from": "Standard.TrackedObject09", "to" : "Actions.TrackedObject09" },
+        { "from": "Standard.TrackedObject10", "to" : "Actions.TrackedObject10" },
+        { "from": "Standard.TrackedObject11", "to" : "Actions.TrackedObject11" },
+        { "from": "Standard.TrackedObject12", "to" : "Actions.TrackedObject12" },
+        { "from": "Standard.TrackedObject13", "to" : "Actions.TrackedObject13" },
+        { "from": "Standard.TrackedObject14", "to" : "Actions.TrackedObject14" },
+        { "from": "Standard.TrackedObject15", "to" : "Actions.TrackedObject15" }
     ]
 }

interface/resources/controllers/vive.json

@@ -1,6 +1,15 @@
 {
     "name": "Vive to Standard",
     "channels": [
+        { "from": "Vive.LY", "to": "Standard.LeftIndexPoint",
+            "peek": true,
+            "filters": [ { "type": "hysteresis", "min": 0.7, "max": 0.75 } ]
+        },
+        { "from": "Vive.RY", "to": "Standard.RightIndexPoint",
+            "peek": true,
+            "filters": [ { "type": "hysteresis", "min": 0.7, "max": 0.75 } ]
+        },
+
         { "from": "Vive.LY", "when": "Vive.LSY", "filters": ["invert"], "to": "Standard.LY" },
         { "from": "Vive.LX", "when": "Vive.LSX", "to": "Standard.LX" },
         {
@@ -13,6 +22,10 @@
 
         { "from": "Vive.LeftGrip", "to": "Standard.LeftGrip" },
         { "from": "Vive.LS", "to": "Standard.LS" },
+        { "from": "Vive.LSTouch", "to": "Standard.LeftThumbUp",
+            "peek": true,
+            "filters": [ { "type": "logicalNot" } ]
+        },
         { "from": "Vive.LSTouch", "to": "Standard.LSTouch" },
 
         { "from": "Vive.RY", "when": "Vive.RSY", "filters": ["invert"], "to": "Standard.RY" },
@@ -27,6 +40,10 @@
 
         { "from": "Vive.RightGrip", "to": "Standard.RightGrip" },
         { "from": "Vive.RS", "to": "Standard.RS" },
+        { "from": "Vive.RSTouch", "to": "Standard.RightThumbUp",
+            "peek": true,
+            "filters": [ { "type": "logicalNot" } ]
+        },
         { "from": "Vive.RSTouch", "to": "Standard.RSTouch" },
 
         { "from": "Vive.LSCenter", "to": "Standard.LeftPrimaryThumb" },
@@ -59,7 +76,24 @@
 
         { "from": "Vive.Head", "to" : "Standard.Head"},
 
-        { "from": "Vive.RightArm", "to" : "Standard.RightArm"},
+        { "from": "Vive.RightArm", "to" : "Standard.RightArm" },
-        { "from": "Vive.LeftArm", "to" : "Standard.LeftArm"}
+        { "from": "Vive.LeftArm", "to" : "Standard.LeftArm" },
+        
+        { "from": "Vive.TrackedObject00", "to" : "Standard.TrackedObject00" },
+        { "from": "Vive.TrackedObject01", "to" : "Standard.TrackedObject01" },
+        { "from": "Vive.TrackedObject02", "to" : "Standard.TrackedObject02" },
+        { "from": "Vive.TrackedObject03", "to" : "Standard.TrackedObject03" },
+        { "from": "Vive.TrackedObject04", "to" : "Standard.TrackedObject04" },
+        { "from": "Vive.TrackedObject05", "to" : "Standard.TrackedObject05" },
+        { "from": "Vive.TrackedObject06", "to" : "Standard.TrackedObject06" },
+        { "from": "Vive.TrackedObject07", "to" : "Standard.TrackedObject07" },
+        { "from": "Vive.TrackedObject08", "to" : "Standard.TrackedObject08" },
+        { "from": "Vive.TrackedObject09", "to" : "Standard.TrackedObject09" },
+        { "from": "Vive.TrackedObject10", "to" : "Standard.TrackedObject10" },
+        { "from": "Vive.TrackedObject11", "to" : "Standard.TrackedObject11" },
+        { "from": "Vive.TrackedObject12", "to" : "Standard.TrackedObject12" },
+        { "from": "Vive.TrackedObject13", "to" : "Standard.TrackedObject13" },
+        { "from": "Vive.TrackedObject14", "to" : "Standard.TrackedObject14" },
+        { "from": "Vive.TrackedObject15", "to" : "Standard.TrackedObject15" }
     ]
 }

interface/resources/qml/AssetServer.qml

@@ -28,6 +28,7 @@ ScrollingWindow {
     minSize: Qt.vector2d(200, 300)
 
     property int colorScheme: hifi.colorSchemes.dark
+    property int selectionMode: SelectionMode.ExtendedSelection
 
     HifiConstants { id: hifi }
 
@@ -35,7 +36,8 @@ ScrollingWindow {
     property var assetProxyModel: Assets.proxyModel;
     property var assetMappingsModel: Assets.mappingModel;
     property var currentDirectory;
-
+    property var selectedItems: treeView.selection.selectedIndexes.length;
+    
     Settings {
         category: "Overlay.AssetServer"
         property alias x: root.x
@@ -48,7 +50,7 @@ ScrollingWindow {
         assetMappingsModel.errorGettingMappings.connect(handleGetMappingsError);
         reload();
     }
-
+    
     function doDeleteFile(path) {
         console.log("Deleting " + path);
 
@@ -118,11 +120,23 @@ ScrollingWindow {
 
     function canAddToWorld(path) {
         var supportedExtensions = [/\.fbx\b/i, /\.obj\b/i];
+        
+        if (selectedItems > 1) {
+            return false;
+        }
 
         return supportedExtensions.reduce(function(total, current) {
             return total | new RegExp(current).test(path);
         }, false);
     }
+    
+    function canRename() {    
+        if (treeView.selection.hasSelection && selectedItems == 1) {
+            return true;
+        } else {
+            return false;
+        }
+    }
 
     function clear() {
         Assets.mappingModel.clear();
@@ -151,13 +165,17 @@ ScrollingWindow {
         var SHAPE_TYPE_SIMPLE_HULL = 1;
         var SHAPE_TYPE_SIMPLE_COMPOUND = 2;
         var SHAPE_TYPE_STATIC_MESH = 3;
-
+        var SHAPE_TYPE_BOX = 4;
+        var SHAPE_TYPE_SPHERE = 5;
+        
         var SHAPE_TYPES = [];
         SHAPE_TYPES[SHAPE_TYPE_NONE] = "No Collision";
         SHAPE_TYPES[SHAPE_TYPE_SIMPLE_HULL] = "Basic - Whole model";
         SHAPE_TYPES[SHAPE_TYPE_SIMPLE_COMPOUND] = "Good - Sub-meshes";
         SHAPE_TYPES[SHAPE_TYPE_STATIC_MESH] = "Exact - All polygons";
-
+        SHAPE_TYPES[SHAPE_TYPE_BOX] = "Box";
+        SHAPE_TYPES[SHAPE_TYPE_SPHERE] = "Sphere";
+        
         var SHAPE_TYPE_DEFAULT = SHAPE_TYPE_STATIC_MESH;
         var DYNAMIC_DEFAULT = false;
         var prompt = desktop.customInputDialog({
@@ -196,6 +214,12 @@ ScrollingWindow {
                     case SHAPE_TYPE_STATIC_MESH:
                         shapeType = "static-mesh";
                         break;
+                    case SHAPE_TYPE_BOX:
+                        shapeType = "box";
+                        break;
+                    case SHAPE_TYPE_SPHERE:
+                        shapeType = "sphere";
+                        break;
                     default:
                         shapeType = "none";
                 }
@@ -289,23 +313,37 @@ ScrollingWindow {
         });
     }
     function deleteFile(index) {
+        var path = [];
+        
         if (!index) {
-            index = treeView.selection.currentIndex;
+            for (var i = 0; i < selectedItems; i++) {
+                 treeView.selection.setCurrentIndex(treeView.selection.selectedIndexes[i], 0x100);
+                 index = treeView.selection.currentIndex;
+                 path[i] = assetProxyModel.data(index, 0x100);                  
+            }
         }
-        var path = assetProxyModel.data(index, 0x100);
+        
         if (!path) {
             return;
         }
-
+        
+        var modalMessage = "";
+        var items = selectedItems.toString();
         var isFolder = assetProxyModel.data(treeView.selection.currentIndex, 0x101);
         var typeString = isFolder ? 'folder' : 'file';
-
+        
+        if (selectedItems > 1) {
+            modalMessage = "You are about to delete " + items + " items \nDo you want to continue?";
+        } else {
+            modalMessage = "You are about to delete the following " + typeString + ":\n" + path + "\nDo you want to continue?";
+        }
+        
         var object = desktop.messageBox({
             icon: hifi.icons.question,
             buttons: OriginalDialogs.StandardButton.Yes + OriginalDialogs.StandardButton.No,
             defaultButton: OriginalDialogs.StandardButton.Yes,
             title: "Delete",
-            text: "You are about to delete the following " + typeString + ":\n" + path + "\nDo you want to continue?"
+            text: modalMessage
         });
         object.selected.connect(function(button) {
             if (button === OriginalDialogs.StandardButton.Yes) {
@@ -445,20 +483,20 @@ ScrollingWindow {
                     color: hifi.buttons.black
                     colorScheme: root.colorScheme
                     width: 120
-
+                    
                     enabled: canAddToWorld(assetProxyModel.data(treeView.selection.currentIndex, 0x100))
-
+                    
                     onClicked: root.addToWorld()
                 }
-
+                
                 HifiControls.Button {
                     text: "Rename"
                     color: hifi.buttons.black
                     colorScheme: root.colorScheme
                     width: 80
-
+                    
                     onClicked: root.renameFile()
-                    enabled: treeView.selection.hasSelection
+                    enabled: canRename()
                 }
 
                 HifiControls.Button {
@@ -514,6 +552,7 @@ ScrollingWindow {
             treeModel: assetProxyModel
             canEdit: true
             colorScheme: root.colorScheme
+            selectionMode: SelectionMode.ExtendedSelection
 
             modifyEl: renameEl
 

interface/resources/qml/controls-uit/RadioButton.qml

@@ -0,0 +1,71 @@
+//
+//  RadioButton.qml
+//
+//  Created by Cain Kilgore on 20th July 2017
+//  Copyright 2017 High Fidelity, Inc.
+//
+//  Distributed under the Apache License, Version 2.0.
+//  See the accompanying file LICENSE or http://www.apache.org/licenses/LICENSE-2.0.html
+//
+
+import QtQuick 2.5
+import QtQuick.Controls 1.4 as Original
+import QtQuick.Controls.Styles 1.4
+
+import "../styles-uit"
+import "../controls-uit" as HifiControls
+
+Original.RadioButton {
+    id: radioButton
+    HifiConstants { id: hifi }
+
+    property int colorScheme: hifi.colorSchemes.light
+    readonly property bool isLightColorScheme: colorScheme == hifi.colorSchemes.light
+
+    readonly property int boxSize: 14
+    readonly property int boxRadius: 3
+    readonly property int checkSize: 10
+    readonly property int checkRadius: 2
+
+    style: RadioButtonStyle {
+        indicator: Rectangle {
+            id: box
+            width: boxSize
+            height: boxSize
+            radius: 7
+            gradient: Gradient {
+                GradientStop {
+                    position: 0.2
+                    color: pressed || hovered
+                           ? (radioButton.isLightColorScheme ? hifi.colors.checkboxDarkStart : hifi.colors.checkboxLightStart)
+                           : (radioButton.isLightColorScheme ? hifi.colors.checkboxLightStart : hifi.colors.checkboxDarkStart)
+                }
+                GradientStop {
+                    position: 1.0
+                    color: pressed || hovered
+                           ? (radioButton.isLightColorScheme ? hifi.colors.checkboxDarkFinish : hifi.colors.checkboxLightFinish)
+                           : (radioButton.isLightColorScheme ? hifi.colors.checkboxLightFinish : hifi.colors.checkboxDarkFinish)
+                }
+            }
+
+            Rectangle {
+                id: check
+                width: checkSize
+                height: checkSize
+                radius: 7
+                anchors.centerIn: parent
+                color: "#00B4EF"
+                border.width: 1
+                border.color: "#36CDFF"
+                visible: checked && !pressed || !checked && pressed
+            }
+        }
+
+        label: RalewaySemiBold {
+            text: control.text
+            size: hifi.fontSizes.inputLabel
+            color: isLightColorScheme ? hifi.colors.lightGray : hifi.colors.lightGrayText
+            x: radioButton.boxSize / 2
+        }
+    }
+}

interface/resources/qml/controls/RadioButton.qml

@@ -0,0 +1,17 @@
+import QtQuick.Controls 1.3 as Original
+import QtQuick.Controls.Styles 1.3
+import "../styles"
+import "."
+Original.RadioButton {
+    text: "Radio Button"
+    style: RadioButtonStyle {
+        indicator: FontAwesome {
+            text: control.checked ? "\uf046" : "\uf096"
+        }
+        label: Text {
+            text: control.text
+            renderType: Text.QtRendering
+        }
+    }
+
+}

interface/resources/qml/dialogs/preferences/PrimaryHandPreference.qml

@@ -0,0 +1,101 @@
+//
+//  PrimaryHandPreference.qml
+//
+//  Created by Cain Kilgore on 20th July 2017
+//  Copyright 2017 High Fidelity, Inc.
+//
+//  Distributed under the Apache License, Version 2.0.
+//  See the accompanying file LICENSE or http://www.apache.org/licenses/LICENSE-2.0.html
+//
+
+import QtQuick 2.5
+
+import "../../controls-uit"
+
+Preference {
+    id: root
+    property alias box1: box1
+    property alias box2: box2
+    
+    height: control.height + hifi.dimensions.controlInterlineHeight
+
+    Component.onCompleted: {
+        if (preference.value == "left") {
+            box1.checked = true;
+        } else {
+            box2.checked = true;
+        }
+    }
+
+    function save() {
+        if (box1.checked && !box2.checked) {
+            preference.value = "left";
+        }
+        if (!box1.checked && box2.checked) {
+            preference.value = "right";
+        }
+        preference.save();
+    }
+
+    Item {
+        id: control
+        anchors {
+            left: parent.left
+            right: parent.right
+            bottom: parent.bottom
+        }
+        height: Math.max(labelName.height, box1.height, box2.height)
+
+        Label {
+            id: labelName
+            text: root.label + ":"
+            colorScheme: hifi.colorSchemes.dark
+            anchors {
+                left: parent.left
+                right: box1.left
+                rightMargin: hifi.dimensions.labelPadding
+                verticalCenter: parent.verticalCenter
+            }
+            horizontalAlignment: Text.AlignRight
+            wrapMode: Text.Wrap
+        }
+
+        RadioButton {
+            id: box1
+            text: "Left"
+            width: 60
+            anchors {
+                right: box2.left
+                verticalCenter: parent.verticalCenter
+            }
+            onClicked: {
+                if (box2.checked) {
+                    box2.checked = false;
+                }
+                if (!box1.checked && !box2.checked) {
+                    box1.checked = true;
+                }
+            }
+            colorScheme: hifi.colorSchemes.dark
+        }
+        
+        RadioButton {
+            id: box2
+            text: "Right"
+            width: 60
+            anchors {
+                right: parent.right
+                verticalCenter: parent.verticalCenter
+            }
+            onClicked: {
+                if (box1.checked) {
+                    box1.checked = false;
+                }
+                if (!box1.checked && !box2.checked) {
+                    box2.checked = true;
+                }
+            }
+            colorScheme: hifi.colorSchemes.dark
+        }
+    }
+}

interface/resources/qml/dialogs/preferences/Section.qml

@@ -73,6 +73,7 @@ Preference {
         property var buttonBuilder: Component { ButtonPreference { } }
         property var comboBoxBuilder: Component { ComboBoxPreference { } }
         property var spinnerSliderBuilder: Component { SpinnerSliderPreference { } }
+        property var primaryHandBuilder: Component { PrimaryHandPreference { } }
         property var preferences: []
         property int checkBoxCount: 0
 
@@ -134,6 +135,11 @@ Preference {
                     checkBoxCount = 0;
                     builder = spinnerSliderBuilder;
                     break;
+                    
+                case Preference.PrimaryHand:
+                    checkBoxCount++;
+                    builder = primaryHandBuilder;
+                    break;
             };
 
             if (builder) {

interface/resources/qml/hifi/audio/Audio.qml

@@ -33,7 +33,7 @@ Rectangle {
 
     // only show the title if loaded through a "loader"
     function showTitle() {
-        return root.parent.objectName == "loader";
+        return (root.parent !== null) && root.parent.objectName == "loader";
     }
 
     Column {

interface/resources/qml/hifi/audio/MicBar.qml

@@ -27,7 +27,7 @@ Rectangle {
 
     color: "#00000000";
     border {
-        width: (standalone || Audio.muted || mouseArea.containsMouse) ? 2 : 0;
+        width: mouseArea.containsMouse || mouseArea.containsPress ? 2 : 0;
         color: colors.border;
     }
 
@@ -61,7 +61,7 @@ Rectangle {
         drag.target: dragTarget;
     }
 
-    Item {
+    QtObject {
         id: colors;
 
         readonly property string unmuted: "#FFF";
@@ -72,7 +72,7 @@ Rectangle {
         readonly property string red: colors.muted;
         readonly property string fill: "#55000000";
         readonly property string border: standalone ? "#80FFFFFF" : "#55FFFFFF";
-        readonly property string icon: (Audio.muted && !mouseArea.containsMouse) ? muted : unmuted;
+        readonly property string icon: Audio.muted ? muted : unmuted;
     }
 
     Item {
@@ -92,10 +92,8 @@ Rectangle {
                 readonly property string unmutedIcon: "../../../icons/tablet-icons/mic-unmute-i.svg";
                 readonly property string mutedIcon: "../../../icons/tablet-icons/mic-mute-i.svg";
 
-                function exclusiveOr(a, b) { return (a || b) && !(a && b); }
-
                 id: image;
-                source: exclusiveOr(Audio.muted, mouseArea.containsMouse) ? mutedIcon : unmutedIcon;
+                source: Audio.muted ? mutedIcon : unmutedIcon;
 
                 width: 30;
                 height: 30;
@@ -118,9 +116,9 @@ Rectangle {
     Item {
         id: status;
 
-        readonly property string color: (Audio.muted && !mouseArea.containsMouse) ? colors.muted : colors.unmuted;
+        readonly property string color: Audio.muted ? colors.muted : colors.unmuted;
 
-        visible: Audio.muted || mouseArea.containsMouse;
+        visible: Audio.muted;
 
         anchors {
             left: parent.left;
@@ -133,14 +131,14 @@ Rectangle {
 
         Text {
             anchors {
-        horizontalCenter: parent.horizontalCenter;
+                horizontalCenter: parent.horizontalCenter;
                 verticalCenter: parent.verticalCenter;
             }
 
             color: parent.color;
 
-            text: Audio.muted ? (mouseArea.containsMouse ? "UNMUTE" : "MUTED") : "MUTE";
+            text: Audio.muted ? "MUTED" : "MUTE";
-        font.pointSize: 12;
+            font.pointSize: 12;
         }
 
         Rectangle {
@@ -150,7 +148,7 @@ Rectangle {
             }
 
             width: 50;
-        height: 4;
+            height: 4;
             color: parent.color;
         }
 
@@ -161,7 +159,7 @@ Rectangle {
             }
 
             width: 50;
-        height: 4;
+            height: 4;
             color: parent.color;
         }
     }

interface/resources/qml/hifi/dialogs/GeneralPreferencesDialog.qml

@@ -17,7 +17,7 @@ PreferencesDialog {
     id: root
     objectName: "GeneralPreferencesDialog"
     title: "General Settings"
-    showCategories: ["UI", "Snapshots", "Scripts", "Privacy", "Octree", "HMD", "Sixense Controllers", "Perception Neuron", "Kinect", "Leap Motion"]
+    showCategories: ["UI", "Snapshots", "Scripts", "Privacy", "Octree", "HMD", "Game Controller", "Sixense Controllers", "Perception Neuron", "Kinect", "Leap Motion"]
     property var settings: Settings {
         category: root.objectName
         property alias x: root.x

interface/resources/qml/hifi/dialogs/TabletAssetServer.qml

@@ -37,6 +37,7 @@ Rectangle {
     property var assetProxyModel: Assets.proxyModel;
     property var assetMappingsModel: Assets.mappingModel;
     property var currentDirectory;
+    property var selectedItems: treeView.selection.selectedIndexes.length;
 
     Settings {
         category: "Overlay.AssetServer"
@@ -119,11 +120,23 @@ Rectangle {
 
     function canAddToWorld(path) {
         var supportedExtensions = [/\.fbx\b/i, /\.obj\b/i];
+        
+        if (selectedItems > 1) {
+            return false;
+        }
 
         return supportedExtensions.reduce(function(total, current) {
             return total | new RegExp(current).test(path);
         }, false);
     }
+    
+    function canRename() {    
+        if (treeView.selection.hasSelection && selectedItems == 1) {
+            return true;
+        } else {
+            return false;
+        }
+    }
 
     function clear() {
         Assets.mappingModel.clear();
@@ -152,13 +165,17 @@ Rectangle {
         var SHAPE_TYPE_SIMPLE_HULL = 1;
         var SHAPE_TYPE_SIMPLE_COMPOUND = 2;
         var SHAPE_TYPE_STATIC_MESH = 3;
-
+        var SHAPE_TYPE_BOX = 4;
+        var SHAPE_TYPE_SPHERE = 5;
+        
         var SHAPE_TYPES = [];
         SHAPE_TYPES[SHAPE_TYPE_NONE] = "No Collision";
         SHAPE_TYPES[SHAPE_TYPE_SIMPLE_HULL] = "Basic - Whole model";
         SHAPE_TYPES[SHAPE_TYPE_SIMPLE_COMPOUND] = "Good - Sub-meshes";
         SHAPE_TYPES[SHAPE_TYPE_STATIC_MESH] = "Exact - All polygons";
-
+        SHAPE_TYPES[SHAPE_TYPE_BOX] = "Box";
+        SHAPE_TYPES[SHAPE_TYPE_SPHERE] = "Sphere";
+        
         var SHAPE_TYPE_DEFAULT = SHAPE_TYPE_STATIC_MESH;
         var DYNAMIC_DEFAULT = false;
         var prompt = tabletRoot.customInputDialog({
@@ -197,6 +214,12 @@ Rectangle {
                 case SHAPE_TYPE_STATIC_MESH:
                     shapeType = "static-mesh";
                     break;
+                case SHAPE_TYPE_BOX:
+                    shapeType = "box";
+                    break;
+                case SHAPE_TYPE_SPHERE:
+                    shapeType = "sphere";
+                    break;
                 default:
                     shapeType = "none";
                 }
@@ -290,23 +313,37 @@ Rectangle {
         });
     }
     function deleteFile(index) {
+        var path = [];
+        
         if (!index) {
-            index = treeView.selection.currentIndex;
+            for (var i = 0; i < selectedItems; i++) {
+                 treeView.selection.setCurrentIndex(treeView.selection.selectedIndexes[i], 0x100);
+                 index = treeView.selection.currentIndex;
+                 path[i] = assetProxyModel.data(index, 0x100);                  
+            }
         }
-        var path = assetProxyModel.data(index, 0x100);
+        
         if (!path) {
             return;
         }
 
+        var modalMessage = "";
+        var items = selectedItems.toString();
         var isFolder = assetProxyModel.data(treeView.selection.currentIndex, 0x101);
         var typeString = isFolder ? 'folder' : 'file';
+        
+        if (selectedItems > 1) {
+            modalMessage = "You are about to delete " + items + " items \nDo you want to continue?";
+        } else {
+            modalMessage = "You are about to delete the following " + typeString + ":\n" + path + "\nDo you want to continue?";
+        }
 
         var object = tabletRoot.messageBox({
                                             icon: hifi.icons.question,
                                             buttons: OriginalDialogs.StandardButton.Yes + OriginalDialogs.StandardButton.No,
                                             defaultButton: OriginalDialogs.StandardButton.Yes,
                                             title: "Delete",
-                                            text: "You are about to delete the following " + typeString + ":\n" + path + "\nDo you want to continue?"
+                                            text: modalMessage
                                         });
         object.selected.connect(function(button) {
             if (button === OriginalDialogs.StandardButton.Yes) {
@@ -459,7 +496,7 @@ Rectangle {
                     width: 80
 
                     onClicked: root.renameFile()
-                    enabled: treeView.selection.hasSelection
+                    enabled: canRename()
                 }
 
                 HifiControls.Button {
@@ -515,6 +552,7 @@ Rectangle {
             treeModel: assetProxyModel
             canEdit: true
             colorScheme: root.colorScheme
+            selectionMode: SelectionMode.ExtendedSelection
 
             modifyEl: renameEl
 

interface/resources/qml/hifi/dialogs/content/AttachmentsContent.qml

@@ -50,7 +50,7 @@ Item {
                         margins: 4
                     }
                     clip: true
-                    snapMode: ListView.SnapToItem
+                    cacheBuffer: 4000
 
                     model: ListModel {}
                     delegate: Item {

interface/resources/qml/hifi/tablet/NewModelDialog.qml

@@ -145,7 +145,9 @@ Rectangle {
                     model: ["No Collision",
                             "Basic - Whole model",
                             "Good - Sub-meshes",
-                            "Exact - All polygons"]
+                            "Exact - All polygons", 
+                            "Box", 
+                            "Sphere"]
                 }
 
                 Row {

interface/resources/qml/hifi/tablet/OpenVrConfiguration.qml

@@ -145,12 +145,13 @@ Rectangle {
         visible: headPuckBox.checked
         HifiControls.SpinBox {
             id: headYOffset
-            decimals: 4
+            decimals: 1
             width: 112
-            label: "Y: offset"
+            label: "Y Offset"
+            suffix: " cm"
             minimumValue: -10
-            stepSize: 0.0254
+            stepSize: 1
-            value: -0.05
+            value: -5
             colorScheme: hifi.colorSchemes.dark
 
             onEditingFinished: {
@@ -162,11 +163,12 @@ Rectangle {
         HifiControls.SpinBox {
             id: headZOffset
             width: 112
-            label: "Z: offset"
+            label: "Z Offset"
             minimumValue: -10
-            stepSize: 0.0254
+            stepSize: 1
-            decimals: 4
+            decimals: 1
-            value: -0.05
+            suffix: " cm"
+            value: -5
             colorScheme: hifi.colorSchemes.dark
 
             onEditingFinished: {
@@ -175,7 +177,6 @@ Rectangle {
         }
     }
 
-
     RalewayBold {
         id: hands
 
@@ -254,11 +255,12 @@ Rectangle {
 
         HifiControls.SpinBox {
             id: handYOffset
-            decimals: 4
+            decimals: 1
             width: 112
-            label: "Y: offset"
+            suffix: " cm"
+            label: "Y Offset"
             minimumValue: -10
-            stepSize: 0.0254
+            stepSize: 1
             colorScheme: hifi.colorSchemes.dark
 
             onEditingFinished: {
@@ -270,10 +272,11 @@ Rectangle {
         HifiControls.SpinBox {
             id: handZOffset
             width: 112
-            label: "Z: offset"
+            label: "Z Offset"
+            suffix: " cm"
             minimumValue: -10
-            stepSize: 0.0254
+            stepSize: 1
-            decimals: 4
+            decimals: 1
             colorScheme: hifi.colorSchemes.dark
 
             onEditingFinished: {
@@ -488,15 +491,55 @@ Rectangle {
         }
     }
 
+    Row {
+        id: shoulderAdditionalConfig
+        visible: shoulderBox.checked
+        anchors.top: shoulderConfig.bottom
+        anchors.topMargin: 5
+        anchors.left: openVrConfiguration.left
+        anchors.leftMargin: leftMargin + 20
+        spacing: 10
+
+        HifiControls.SpinBox {
+            id: armCircumference
+            decimals: 1
+            width: 160
+            suffix: " cm"
+            label: "Arm Circumference"
+            minimumValue: 0
+            stepSize: 1.0
+            colorScheme: hifi.colorSchemes.dark
+            value: 33.0
+
+            onEditingFinished: {
+                sendConfigurationSettings();
+            }
+        }
+
+        HifiControls.SpinBox {
+            id: shoulderWidth
+            width: 160
+            label: "Shoulder Width"
+            suffix: " cm"
+            minimumValue: 0
+            stepSize: 1.0
+            decimals: 1
+            colorScheme: hifi.colorSchemes.dark
+            value: 48
+
+            onEditingFinished: {
+                sendConfigurationSettings();
+            }
+        }
+    }
+
     Separator {
         id: bottomSeperator
         width: parent.width
-        anchors.top: shoulderConfig.bottom
+        anchors.top: shoulderAdditionalConfig.visible ? shoulderAdditionalConfig.bottom : shoulderConfig.bottom
-        anchors.topMargin: 10
+        anchors.topMargin: (shoulderAdditionalConfig.visible ? 25 : 10)
     }
 
-
-
     Rectangle {
         id: calibrationButton
         property int color: hifi.buttons.blue
@@ -835,6 +878,9 @@ Rectangle {
         var viveController = settings["handController"];
         var desktopMode = settings["desktopMode"];
 
+        armCircumference.value = settings.armCircumference;
+        shoulderWidth.value = settings.shoulderWidth;
+
         if (HmdHead) {
             headBox.checked = true;
             headPuckBox.checked = false;
@@ -1010,6 +1056,8 @@ Rectangle {
             "bodyConfiguration": trackerConfiguration,
             "headConfiguration": headObject,
             "handConfiguration": handObject,
+            "armCircumference": armCircumference.value,
+            "shoulderWidth": shoulderWidth.value,
             "desktopMode": viveInDesktop.checked
         }
 

interface/resources/qml/hifi/tablet/TabletGeneralPreferences.qml

@@ -32,6 +32,6 @@ StackView {
     TabletPreferencesDialog {
         id: root
         objectName: "TabletGeneralPreferences"
-        showCategories: ["UI", "Snapshots", "Scripts", "Privacy", "Octree", "HMD", "Sixense Controllers", "Perception Neuron", "Kinect", "Vive Pucks Configuration", "Leap Motion"]
+        showCategories: ["UI", "Snapshots", "Scripts", "Privacy", "Octree", "HMD", "Game Controller", "Sixense Controllers", "Perception Neuron", "Kinect", "Leap Motion"]
     }
 }

interface/resources/qml/hifi/tablet/TabletMenuStack.qml

@@ -114,6 +114,7 @@ Item {
         }
 
         function clearMenus() {
+            topMenu = null
             d.clear()
         }
 

interface/resources/qml/hifi/tablet/TabletRoot.qml

@@ -202,5 +202,11 @@ Item {
     width: 480
     height: 706
 
-    function setShown(value) {}
+    function setShown(value) {
+        if (value === true) {
+            HMD.openTablet()
+        } else {
+            HMD.closeTablet()
+        }
+    }
 }

interface/resources/qml/hifi/tablet/tabletWindows/preferences/Section.qml

@@ -82,6 +82,7 @@ Preference {
         property var buttonBuilder: Component { ButtonPreference { } }
         property var comboBoxBuilder: Component { ComboBoxPreference { } }
         property var spinnerSliderBuilder: Component { SpinnerSliderPreference { } }
+        property var primaryHandBuilder: Component { PrimaryHandPreference { } }
         property var preferences: []
         property int checkBoxCount: 0
 
@@ -144,10 +145,16 @@ Preference {
                     //to be not overlapped when drop down is active
                     zpos = root.z + 1000 - itemNum
                     break;
+
                 case Preference.SpinnerSlider:
                     checkBoxCount = 0;
                     builder = spinnerSliderBuilder;
                     break;
+
+                case Preference.PrimaryHand:
+                    checkBoxCount++;
+                    builder = primaryHandBuilder;
+                    break;
             };
 
             if (builder) {

interface/src/Application.cpp

@@ -61,6 +61,7 @@
 #include <AssetClient.h>
 #include <AssetUpload.h>
 #include <AutoUpdater.h>
+#include <Midi.h>
 #include <AudioInjectorManager.h>
 #include <AvatarBookmarks.h>
 #include <CursorManager.h>
@@ -69,7 +70,7 @@
 #include <EntityScriptClient.h>
 #include <EntityScriptServerLogClient.h>
 #include <EntityScriptingInterface.h>
-#include <HoverOverlayInterface.h>
+#include "ui/overlays/ContextOverlayInterface.h"
 #include <ErrorDialog.h>
 #include <FileScriptingInterface.h>
 #include <Finally.h>
@@ -228,6 +229,7 @@ static const QString FBX_EXTENSION  = ".fbx";
 static const QString OBJ_EXTENSION  = ".obj";
 static const QString AVA_JSON_EXTENSION = ".ava.json";
 static const QString WEB_VIEW_TAG = "noDownload=true";
+static const QString ZIP_EXTENSION = ".zip";
 
 static const float MIRROR_FULLSCREEN_DISTANCE = 0.389f;
 
@@ -259,7 +261,8 @@ const QHash<QString, Application::AcceptURLMethod> Application::_acceptedExtensi
     { AVA_JSON_EXTENSION, &Application::askToWearAvatarAttachmentUrl },
     { JSON_EXTENSION, &Application::importJSONFromURL },
     { JS_EXTENSION, &Application::askToLoadScript },
-    { FST_EXTENSION, &Application::askToSetAvatarUrl }
+    { FST_EXTENSION, &Application::askToSetAvatarUrl },
+    { ZIP_EXTENSION, &Application::importFromZIP }
 };
 
 class DeadlockWatchdogThread : public QThread {
@@ -398,6 +401,10 @@ public:
                     return true;
                 }
             }
+
+            if (message->message == WM_DEVICECHANGE) {
+                Midi::USBchanged();                // re-scan the MIDI bus
+            }
         }
         return false;
     }
@@ -567,6 +574,7 @@ bool setupEssentials(int& argc, char** argv, bool runningMarkerExisted) {
     DependencyManager::set<SceneScriptingInterface>();
     DependencyManager::set<OffscreenUi>();
     DependencyManager::set<AutoUpdater>();
+    DependencyManager::set<Midi>();
     DependencyManager::set<PathUtils>();
     DependencyManager::set<InterfaceDynamicFactory>();
     DependencyManager::set<AudioInjectorManager>();
@@ -589,7 +597,7 @@ bool setupEssentials(int& argc, char** argv, bool runningMarkerExisted) {
     DependencyManager::set<Snapshot>();
     DependencyManager::set<CloseEventSender>();
     DependencyManager::set<ResourceManager>();
-    DependencyManager::set<HoverOverlayInterface>();
+    DependencyManager::set<ContextOverlayInterface>();
 
     return previousSessionCrashed;
 }
@@ -618,6 +626,7 @@ const float DEFAULT_DESKTOP_TABLET_SCALE_PERCENT = 75.0f;
 const bool DEFAULT_DESKTOP_TABLET_BECOMES_TOOLBAR = true;
 const bool DEFAULT_HMD_TABLET_BECOMES_TOOLBAR = false;
 const bool DEFAULT_PREFER_AVATAR_FINGER_OVER_STYLUS = false;
+const QString DEFAULT_CURSOR_NAME = "DEFAULT";
 
 Application::Application(int& argc, char** argv, QElapsedTimer& startupTimer, bool runningMarkerExisted) :
     QApplication(argc, argv),
@@ -637,6 +646,7 @@ Application::Application(int& argc, char** argv, QElapsedTimer& startupTimer, bo
     _hmdTabletBecomesToolbarSetting("hmdTabletBecomesToolbar", DEFAULT_HMD_TABLET_BECOMES_TOOLBAR),
     _preferAvatarFingerOverStylusSetting("preferAvatarFingerOverStylus", DEFAULT_PREFER_AVATAR_FINGER_OVER_STYLUS),
     _constrainToolbarPosition("toolbar/constrainToolbarToCenterX", true),
+    _preferredCursor("preferredCursor", DEFAULT_CURSOR_NAME),
     _scaleMirror(1.0f),
     _rotateMirror(0.0f),
     _raiseMirror(0.0f),
@@ -932,14 +942,7 @@ Application::Application(int& argc, char** argv, QElapsedTimer& startupTimer, bo
     _glWidget->setFocusPolicy(Qt::StrongFocus);
     _glWidget->setFocus();
 
-#ifdef Q_OS_MAC
+    showCursor(Cursor::Manager::lookupIcon(_preferredCursor.get()));
-    auto cursorTarget = _window; // OSX doesn't seem to provide for hiding the cursor only on the GL widget
-#else
-    // On windows and linux, hiding the top level cursor also means it's invisible when hovering over the
-    // window menu, which is a pain, so only hide it for the GL surface
-    auto cursorTarget = _glWidget;
-#endif
-    cursorTarget->setCursor(Qt::BlankCursor);
 
     // enable mouse tracking; otherwise, we only get drag events
     _glWidget->setMouseTracking(true);
@@ -1323,12 +1326,16 @@ Application::Application(int& argc, char** argv, QElapsedTimer& startupTimer, bo
     // Keyboard focus handling for Web overlays.
     auto overlays = &(qApp->getOverlays());
 
-    connect(overlays, &Overlays::mousePressOnOverlay, [=](OverlayID overlayID, const PointerEvent& event) {
+    connect(overlays, &Overlays::mousePressOnOverlay, [=](const OverlayID& overlayID, const PointerEvent& event) {
-        setKeyboardFocusEntity(UNKNOWN_ENTITY_ID);
+        auto thisOverlay = std::dynamic_pointer_cast<Web3DOverlay>(overlays->getOverlay(overlayID));
-        setKeyboardFocusOverlay(overlayID);
+        // Only Web overlays can have keyboard focus.
+        if (thisOverlay) {
+            setKeyboardFocusEntity(UNKNOWN_ENTITY_ID);
+            setKeyboardFocusOverlay(overlayID);
+        }
     });
 
-    connect(overlays, &Overlays::overlayDeleted, [=](OverlayID overlayID) {
+    connect(overlays, &Overlays::overlayDeleted, [=](const OverlayID& overlayID) {
         if (overlayID == _keyboardFocusedOverlay.get()) {
             setKeyboardFocusOverlay(UNKNOWN_OVERLAY_ID);
         }
@@ -1343,6 +1350,21 @@ Application::Application(int& argc, char** argv, QElapsedTimer& startupTimer, bo
         setKeyboardFocusEntity(UNKNOWN_ENTITY_ID);
     });
 
+    connect(overlays,
+        SIGNAL(mousePressOnOverlay(const OverlayID&, const PointerEvent&)),
+        DependencyManager::get<ContextOverlayInterface>().data(),
+        SLOT(contextOverlays_mousePressOnOverlay(const OverlayID&, const PointerEvent&)));
+
+    connect(overlays,
+        SIGNAL(hoverEnterOverlay(const OverlayID&, const PointerEvent&)),
+        DependencyManager::get<ContextOverlayInterface>().data(),
+        SLOT(contextOverlays_hoverEnterOverlay(const OverlayID&, const PointerEvent&)));
+
+    connect(overlays,
+        SIGNAL(hoverLeaveOverlay(const OverlayID&, const PointerEvent&)),
+        DependencyManager::get<ContextOverlayInterface>().data(),
+        SLOT(contextOverlays_hoverLeaveOverlay(const OverlayID&, const PointerEvent&)));
+
     // Add periodic checks to send user activity data
     static int CHECK_NEARBY_AVATARS_INTERVAL_MS = 10000;
     static int NEARBY_AVATAR_RADIUS_METERS = 10;
@@ -1469,7 +1491,7 @@ Application::Application(int& argc, char** argv, QElapsedTimer& startupTimer, bo
         properties["atp_mapping_requests"] = atpMappingRequests;
 
         properties["throttled"] = _displayPlugin ? _displayPlugin->isThrottled() : false;
-        
+
         QJsonObject bytesDownloaded;
         bytesDownloaded["atp"] = statTracker->getStat(STAT_ATP_RESOURCE_TOTAL_BYTES).toInt();
         bytesDownloaded["http"] = statTracker->getStat(STAT_HTTP_RESOURCE_TOTAL_BYTES).toInt();
@@ -1737,9 +1759,16 @@ void Application::checkChangeCursor() {
     }
 }
 
-void Application::showCursor(const QCursor& cursor) {
+void Application::showCursor(const Cursor::Icon& cursor) {
     QMutexLocker locker(&_changeCursorLock);
-    _desiredCursor = cursor;
+
+    auto managedCursor = Cursor::Manager::instance().getCursor();
+    auto curIcon = managedCursor->getIcon();
+    if (curIcon != cursor) {
+        managedCursor->setIcon(cursor);
+        curIcon = cursor;
+    }
+    _desiredCursor = cursor == Cursor::Icon::SYSTEM ? Qt::ArrowCursor : Qt::BlankCursor;
     _cursorNeedsChanging = true;
 }
 
@@ -2114,7 +2143,6 @@ void Application::initializeUi() {
     surfaceContext->setContextProperty("AvatarManager", DependencyManager::get<AvatarManager>().data());
     surfaceContext->setContextProperty("UndoStack", &_undoStackScriptingInterface);
     surfaceContext->setContextProperty("LODManager", DependencyManager::get<LODManager>().data());
-    surfaceContext->setContextProperty("Paths", DependencyManager::get<PathUtils>().data());
     surfaceContext->setContextProperty("HMD", DependencyManager::get<HMDScriptingInterface>().data());
     surfaceContext->setContextProperty("Scene", DependencyManager::get<SceneScriptingInterface>().data());
     surfaceContext->setContextProperty("Render", _renderEngine->getConfiguration().get());
@@ -2124,7 +2152,7 @@ void Application::initializeUi() {
     surfaceContext->setContextProperty("ApplicationCompositor", &getApplicationCompositor());
 
     surfaceContext->setContextProperty("AvatarInputs", AvatarInputs::getInstance());
-    surfaceContext->setContextProperty("HoverOverlay", DependencyManager::get<HoverOverlayInterface>().data());
+    surfaceContext->setContextProperty("ContextOverlay", DependencyManager::get<ContextOverlayInterface>().data());
 
     if (auto steamClient = PluginManager::getInstance()->getSteamClientPlugin()) {
         surfaceContext->setContextProperty("Steam", new SteamScriptingInterface(engine, steamClient.get()));
@@ -2160,9 +2188,11 @@ void Application::initializeUi() {
     _window->setMenuBar(new Menu());
 
     auto compositorHelper = DependencyManager::get<CompositorHelper>();
-    connect(compositorHelper.data(), &CompositorHelper::allowMouseCaptureChanged, [=] {
+    connect(compositorHelper.data(), &CompositorHelper::allowMouseCaptureChanged, this, [=] {
         if (isHMDMode()) {
-            showCursor(compositorHelper->getAllowMouseCapture() ? Qt::BlankCursor : Qt::ArrowCursor);
+            showCursor(compositorHelper->getAllowMouseCapture() ?
+                       Cursor::Manager::lookupIcon(_preferredCursor.get()) :
+                       Cursor::Icon::SYSTEM);
         }
     });
 
@@ -2228,7 +2258,7 @@ void Application::paintGL() {
             QMutexLocker viewLocker(&_viewMutex);
             _viewFrustum.calculate();
         }
-        renderArgs = RenderArgs(_gpuContext, getEntities(), lodManager->getOctreeSizeScale(),
+        renderArgs = RenderArgs(_gpuContext, lodManager->getOctreeSizeScale(),
             lodManager->getBoundaryLevelAdjust(), RenderArgs::DEFAULT_RENDER_MODE,
             RenderArgs::MONO, RenderArgs::RENDER_DEBUG_NONE);
         {
@@ -2314,7 +2344,7 @@ void Application::paintGL() {
                 }
             } else if (_myCamera.getMode() == CAMERA_MODE_MIRROR) {
                 if (isHMDMode()) {
-                    auto mirrorBodyOrientation = myAvatar->getWorldAlignedOrientation() * glm::quat(glm::vec3(0.0f, PI + _rotateMirror, 0.0f));
+                    auto mirrorBodyOrientation = myAvatar->getOrientation() * glm::quat(glm::vec3(0.0f, PI + _rotateMirror, 0.0f));
 
                     glm::quat hmdRotation = extractRotation(myAvatar->getHMDSensorMatrix());
                     // Mirror HMD yaw and roll
@@ -2336,7 +2366,7 @@ void Application::paintGL() {
                         + mirrorBodyOrientation * glm::vec3(0.0f, 0.0f, 1.0f) * MIRROR_FULLSCREEN_DISTANCE * _scaleMirror
                         + mirrorBodyOrientation * hmdOffset);
                 } else {
-                    _myCamera.setOrientation(myAvatar->getWorldAlignedOrientation()
+                    _myCamera.setOrientation(myAvatar->getOrientation()
                         * glm::quat(glm::vec3(0.0f, PI + _rotateMirror, 0.0f)));
                     _myCamera.setPosition(myAvatar->getDefaultEyePosition()
                         + glm::vec3(0, _raiseMirror * myAvatar->getUniformScale(), 0)
@@ -2504,6 +2534,12 @@ void Application::setPreferAvatarFingerOverStylus(bool value) {
     _preferAvatarFingerOverStylusSetting.set(value);
 }
 
+void Application::setPreferredCursor(const QString& cursorName) {
+    qCDebug(interfaceapp) << "setPreferredCursor" << cursorName;
+    _preferredCursor.set(cursorName.isEmpty() ? DEFAULT_CURSOR_NAME : cursorName);
+    showCursor(Cursor::Manager::lookupIcon(_preferredCursor.get()));
+}
+
 void Application::setSettingConstrainToolbarPosition(bool setting) {
     _constrainToolbarPosition.set(setting);
     DependencyManager::get<OffscreenUi>()->setConstrainToolbarToCenterX(setting);
@@ -2738,6 +2774,20 @@ bool Application::importSVOFromURL(const QString& urlString) {
     return true;
 }
 
+void Application::onPresent(quint32 frameCount) {
+    if (shouldPaint()) {
+        postEvent(this, new QEvent(static_cast<QEvent::Type>(Idle)), Qt::HighEventPriority);
+        postEvent(this, new QEvent(static_cast<QEvent::Type>(Paint)), Qt::HighEventPriority);
+    }
+}
+
+bool Application::importFromZIP(const QString& filePath) {
+    qDebug() << "A zip file has been dropped in: " << filePath;
+    QUrl empty;
+    qApp->getFileDownloadInterface()->runUnzip(filePath, empty, true, true);
+    return true;
+}
+
 bool _renderRequested { false };
 
 bool Application::event(QEvent* event) {
@@ -2754,23 +2804,9 @@ bool Application::event(QEvent* event) {
         // Explicit idle keeps the idle running at a lower interval, but without any rendering
         // see (windowMinimizedChanged)
         case Event::Idle:
-            {
+            idle();
-                float nsecsElapsed = (float)_lastTimeUpdated.nsecsElapsed();
+            // Clear the event queue of pending idle calls
-                _lastTimeUpdated.start();
+            removePostedEvents(this, Idle);
-                idle(nsecsElapsed);
-            }
-            return true;
-
-        case Event::Present:
-            if (!_renderRequested) {
-                float nsecsElapsed = (float)_lastTimeUpdated.nsecsElapsed();
-                if (shouldPaint(nsecsElapsed)) {
-                    _renderRequested = true;
-                    _lastTimeUpdated.start();
-                    idle(nsecsElapsed);
-                    postEvent(this, new QEvent(static_cast<QEvent::Type>(Paint)), Qt::HighEventPriority);
-                }
-            }
             return true;
 
         case Event::Paint:
@@ -2778,9 +2814,8 @@ bool Application::event(QEvent* event) {
             //       or AvatarInputs will mysteriously move to the bottom-right
             AvatarInputs::getInstance()->update();
             paintGL();
-            // wait for the next present event before starting idle / paint again
+            // Clear the event queue of pending paint calls
-            removePostedEvents(this, Present);
+            removePostedEvents(this, Paint);
-            _renderRequested = false;
             return true;
 
         default:
@@ -3020,7 +3055,9 @@ void Application::keyPressEvent(QKeyEvent* event) {
                 break;
 
             case Qt::Key_F: {
-                _physicsEngine->dumpNextStats();
+                if (isOption) {
+                    _physicsEngine->dumpNextStats();
+                }
                 break;
             }
 
@@ -3064,9 +3101,13 @@ void Application::keyPressEvent(QKeyEvent* event) {
                     auto cursor = Cursor::Manager::instance().getCursor();
                     auto curIcon = cursor->getIcon();
                     if (curIcon == Cursor::Icon::DEFAULT) {
-                        cursor->setIcon(Cursor::Icon::LINK);
+                        showCursor(Cursor::Icon::RETICLE);
+                    } else if (curIcon == Cursor::Icon::RETICLE) {
+                        showCursor(Cursor::Icon::SYSTEM);
+                    } else if (curIcon == Cursor::Icon::SYSTEM) {
+                        showCursor(Cursor::Icon::LINK);
                     } else {
-                        cursor->setIcon(Cursor::Icon::DEFAULT);
+                        showCursor(Cursor::Icon::DEFAULT);
                     }
                 } else {
                     resetSensors(true);
@@ -3593,7 +3634,7 @@ bool Application::acceptSnapshot(const QString& urlString) {
 
 static uint32_t _renderedFrameIndex { INVALID_FRAME };
 
-bool Application::shouldPaint(float nsecsElapsed) {
+bool Application::shouldPaint() {
     if (_aboutToQuit) {
         return false;
     }
@@ -3613,11 +3654,9 @@ bool Application::shouldPaint(float nsecsElapsed) {
             (float)paintDelaySamples / paintDelayUsecs << "us";
     }
 #endif
-
+    
-    float msecondsSinceLastUpdate = nsecsElapsed / NSECS_PER_USEC / USECS_PER_MSEC;
-
     // Throttle if requested
-    if (displayPlugin->isThrottled() && (msecondsSinceLastUpdate < THROTTLED_SIM_FRAME_PERIOD_MS)) {
+    if (displayPlugin->isThrottled() && (_lastTimeUpdated.elapsed() < THROTTLED_SIM_FRAME_PERIOD_MS)) {
         return false;
     }
 
@@ -3834,7 +3873,7 @@ void setupCpuMonitorThread() {
 #endif
 
 
-void Application::idle(float nsecsElapsed) {
+void Application::idle() {
     PerformanceTimer perfTimer("idle");
 
     // Update the deadlock watchdog
@@ -3891,7 +3930,8 @@ void Application::idle(float nsecsElapsed) {
         steamClient->runCallbacks();
     }
 
-    float secondsSinceLastUpdate = nsecsElapsed / NSECS_PER_MSEC / MSECS_PER_SECOND;
+    float secondsSinceLastUpdate = (float)_lastTimeUpdated.nsecsElapsed() / NSECS_PER_MSEC / MSECS_PER_SECOND;
+    _lastTimeUpdated.start();
 
     // If the offscreen Ui has something active that is NOT the root, then assume it has keyboard focus.
     if (_keyboardDeviceHasFocus && offscreenUi && offscreenUi->getWindow()->activeFocusItem() != offscreenUi->getRootItem()) {
@@ -4125,6 +4165,7 @@ void Application::loadSettings() {
     //DependencyManager::get<LODManager>()->setAutomaticLODAdjust(false);
 
     Menu::getInstance()->loadSettings();
+
     // If there is a preferred plugin, we probably messed it up with the menu settings, so fix it.
     auto pluginManager = PluginManager::getInstance();
     auto plugins = pluginManager->getPreferredDisplayPlugins();
@@ -4138,24 +4179,44 @@ void Application::loadSettings() {
                 break;
             }
         }
-    } else {
+    }
+
+    Setting::Handle<bool> firstRun { Settings::firstRun, true };
+    bool isFirstPerson = false;
+    if (firstRun.get()) {
         // If this is our first run, and no preferred devices were set, default to
         // an HMD device if available.
-        Setting::Handle<bool> firstRun { Settings::firstRun, true };
+        auto displayPlugins = pluginManager->getDisplayPlugins();
-        if (firstRun.get()) {
+        for (auto& plugin : displayPlugins) {
-            auto displayPlugins = pluginManager->getDisplayPlugins();
+            if (plugin->isHmd()) {
-            for (auto& plugin : displayPlugins) {
+                if (auto action = menu->getActionForOption(plugin->getName())) {
-                if (plugin->isHmd()) {
+                    action->setChecked(true);
-                    if (auto action = menu->getActionForOption(plugin->getName())) {
+                    action->trigger();
-                        action->setChecked(true);
+                    break;
-                        action->trigger();
-                        break;
-                    }
                 }
             }
         }
+
+        isFirstPerson = (qApp->isHMDMode());
+    } else {
+        // if this is not the first run, the camera will be initialized differently depending on user settings
+
+        if (qApp->isHMDMode()) {
+            // if the HMD is active, use first-person camera, unless the appropriate setting is checked
+            isFirstPerson = menu->isOptionChecked(MenuOption::FirstPersonHMD);
+        } else {
+            // if HMD is not active, only use first person if the menu option is checked
+            isFirstPerson = menu->isOptionChecked(MenuOption::FirstPerson);
+        }
     }
 
+    // finish initializing the camera, based on everything we checked above. Third person camera will be used if no settings
+    // dictated that we should be in first person
+    Menu::getInstance()->setIsOptionChecked(MenuOption::FirstPerson, isFirstPerson);
+    Menu::getInstance()->setIsOptionChecked(MenuOption::ThirdPerson, !isFirstPerson);
+    _myCamera.setMode((isFirstPerson) ? CAMERA_MODE_FIRST_PERSON : CAMERA_MODE_THIRD_PERSON);
+    cameraMenuChanged();
+
     auto inputs = pluginManager->getInputPlugins();
     for (auto plugin : inputs) {
         if (!plugin->isActive()) {
@@ -4204,7 +4265,6 @@ void Application::init() {
     DependencyManager::get<DeferredLightingEffect>()->init();
 
     DependencyManager::get<AvatarManager>()->init();
-    _myCamera.setMode(CAMERA_MODE_FIRST_PERSON);
 
     _timerStart.start();
     _lastTimeUpdated.start();
@@ -4356,10 +4416,9 @@ void Application::updateMyAvatarLookAtPosition() {
             }
         } else {
             //  I am not looking at anyone else, so just look forward
-            auto headPose = myAvatar->getHeadControllerPoseInSensorFrame();
+            auto headPose = myAvatar->getControllerPoseInWorldFrame(controller::Action::HEAD);
             if (headPose.isValid()) {
-                glm::mat4 worldHeadMat = myAvatar->getSensorToWorldMatrix() * headPose.getMatrix();
+                lookAtSpot = transformPoint(headPose.getMatrix(), glm::vec3(0.0f, 0.0f, TREE_SCALE));
-                lookAtSpot = transformPoint(worldHeadMat, glm::vec3(0.0f, 0.0f, TREE_SCALE));
             } else {
                 lookAtSpot = myAvatar->getHead()->getEyePosition() +
                     (myAvatar->getHead()->getFinalOrientationInWorldFrame() * glm::vec3(0.0f, 0.0f, -TREE_SCALE));
@@ -4460,27 +4519,29 @@ void Application::cameraModeChanged() {
 
 
 void Application::cameraMenuChanged() {
-    if (Menu::getInstance()->isOptionChecked(MenuOption::FullscreenMirror)) {
+    auto menu = Menu::getInstance();
+    if (menu->isOptionChecked(MenuOption::FullscreenMirror)) {
         if (_myCamera.getMode() != CAMERA_MODE_MIRROR) {
             _myCamera.setMode(CAMERA_MODE_MIRROR);
+            getMyAvatar()->reset(false, false, false); // to reset any active MyAvatar::FollowHelpers
         }
-    } else if (Menu::getInstance()->isOptionChecked(MenuOption::FirstPerson)) {
+    } else if (menu->isOptionChecked(MenuOption::FirstPerson)) {
         if (_myCamera.getMode() != CAMERA_MODE_FIRST_PERSON) {
             _myCamera.setMode(CAMERA_MODE_FIRST_PERSON);
             getMyAvatar()->setBoomLength(MyAvatar::ZOOM_MIN);
         }
-    } else if (Menu::getInstance()->isOptionChecked(MenuOption::ThirdPerson)) {
+    } else if (menu->isOptionChecked(MenuOption::ThirdPerson)) {
         if (_myCamera.getMode() != CAMERA_MODE_THIRD_PERSON) {
             _myCamera.setMode(CAMERA_MODE_THIRD_PERSON);
             if (getMyAvatar()->getBoomLength() == MyAvatar::ZOOM_MIN) {
                 getMyAvatar()->setBoomLength(MyAvatar::ZOOM_DEFAULT);
             }
         }
-    } else if (Menu::getInstance()->isOptionChecked(MenuOption::IndependentMode)) {
+    } else if (menu->isOptionChecked(MenuOption::IndependentMode)) {
         if (_myCamera.getMode() != CAMERA_MODE_INDEPENDENT) {
             _myCamera.setMode(CAMERA_MODE_INDEPENDENT);
         }
-    } else if (Menu::getInstance()->isOptionChecked(MenuOption::CameraEntityMode)) {
+    } else if (menu->isOptionChecked(MenuOption::CameraEntityMode)) {
         if (_myCamera.getMode() != CAMERA_MODE_ENTITY) {
             _myCamera.setMode(CAMERA_MODE_ENTITY);
         }
@@ -4771,52 +4832,76 @@ void Application::update(float deltaTime) {
         myAvatar->setDriveKey(MyAvatar::ZOOM, userInputMapper->getActionState(controller::Action::TRANSLATE_CAMERA_Z));
     }
 
-    controller::Pose leftHandPose = userInputMapper->getPoseState(controller::Action::LEFT_HAND);
+    static const std::vector<controller::Action> avatarControllerActions = {
-    controller::Pose rightHandPose = userInputMapper->getPoseState(controller::Action::RIGHT_HAND);
+        controller::Action::LEFT_HAND,
-    auto myAvatarMatrix = createMatFromQuatAndPos(myAvatar->getOrientation(), myAvatar->getPosition());
+        controller::Action::RIGHT_HAND,
-    auto worldToSensorMatrix = glm::inverse(myAvatar->getSensorToWorldMatrix());
+        controller::Action::LEFT_FOOT,
-    auto avatarToSensorMatrix = worldToSensorMatrix * myAvatarMatrix;
+        controller::Action::RIGHT_FOOT,
-    myAvatar->setHandControllerPosesInSensorFrame(leftHandPose.transform(avatarToSensorMatrix), rightHandPose.transform(avatarToSensorMatrix));
+        controller::Action::HIPS,
+        controller::Action::SPINE2,
+        controller::Action::HEAD,
+        controller::Action::LEFT_HAND_THUMB1,
+        controller::Action::LEFT_HAND_THUMB2,
+        controller::Action::LEFT_HAND_THUMB3,
+        controller::Action::LEFT_HAND_THUMB4,
+        controller::Action::LEFT_HAND_INDEX1,
+        controller::Action::LEFT_HAND_INDEX2,
+        controller::Action::LEFT_HAND_INDEX3,
+        controller::Action::LEFT_HAND_INDEX4,
+        controller::Action::LEFT_HAND_MIDDLE1,
+        controller::Action::LEFT_HAND_MIDDLE2,
+        controller::Action::LEFT_HAND_MIDDLE3,
+        controller::Action::LEFT_HAND_MIDDLE4,
+        controller::Action::LEFT_HAND_RING1,
+        controller::Action::LEFT_HAND_RING2,
+        controller::Action::LEFT_HAND_RING3,
+        controller::Action::LEFT_HAND_RING4,
+        controller::Action::LEFT_HAND_PINKY1,
+        controller::Action::LEFT_HAND_PINKY2,
+        controller::Action::LEFT_HAND_PINKY3,
+        controller::Action::LEFT_HAND_PINKY4,
+        controller::Action::RIGHT_HAND_THUMB1,
+        controller::Action::RIGHT_HAND_THUMB2,
+        controller::Action::RIGHT_HAND_THUMB3,
+        controller::Action::RIGHT_HAND_THUMB4,
+        controller::Action::RIGHT_HAND_INDEX1,
+        controller::Action::RIGHT_HAND_INDEX2,
+        controller::Action::RIGHT_HAND_INDEX3,
+        controller::Action::RIGHT_HAND_INDEX4,
+        controller::Action::RIGHT_HAND_MIDDLE1,
+        controller::Action::RIGHT_HAND_MIDDLE2,
+        controller::Action::RIGHT_HAND_MIDDLE3,
+        controller::Action::RIGHT_HAND_MIDDLE4,
+        controller::Action::RIGHT_HAND_RING1,
+        controller::Action::RIGHT_HAND_RING2,
+        controller::Action::RIGHT_HAND_RING3,
+        controller::Action::RIGHT_HAND_RING4,
+        controller::Action::RIGHT_HAND_PINKY1,
+        controller::Action::RIGHT_HAND_PINKY2,
+        controller::Action::RIGHT_HAND_PINKY3,
+        controller::Action::RIGHT_HAND_PINKY4,
+        controller::Action::LEFT_ARM,
+        controller::Action::RIGHT_ARM,
+        controller::Action::LEFT_SHOULDER,
+        controller::Action::RIGHT_SHOULDER,
+        controller::Action::LEFT_FORE_ARM,
+        controller::Action::RIGHT_FORE_ARM,
+        controller::Action::LEFT_LEG,
+        controller::Action::RIGHT_LEG,
+        controller::Action::LEFT_UP_LEG,
+        controller::Action::RIGHT_UP_LEG,
+        controller::Action::LEFT_TOE_BASE,
+        controller::Action::RIGHT_TOE_BASE
+    };
 
-    // If have previously done finger poses or there are new valid finger poses, update finger pose values. This so that if
+    // copy controller poses from userInputMapper to myAvatar.
-    // fingers are not being controlled, finger joints are not updated in MySkeletonModel.
+    glm::mat4 myAvatarMatrix = createMatFromQuatAndPos(myAvatar->getOrientation(), myAvatar->getPosition());
-    // Assumption: Finger poses are either all present and valid or not present at all; thus can test just one joint.
+    glm::mat4 worldToSensorMatrix = glm::inverse(myAvatar->getSensorToWorldMatrix());
-    MyAvatar::FingerPosesMap leftHandFingerPoses;
+    glm::mat4 avatarToSensorMatrix = worldToSensorMatrix * myAvatarMatrix;
-    if (myAvatar->getLeftHandFingerControllerPosesInSensorFrame().size() > 0
+    for (auto& action : avatarControllerActions) {
-            || userInputMapper->getPoseState(controller::Action::LEFT_HAND_THUMB1).isValid()) {
+        controller::Pose pose = userInputMapper->getPoseState(action);
-        for (int i = (int)controller::Action::LEFT_HAND_THUMB1; i <= (int)controller::Action::LEFT_HAND_PINKY4; i++) {
+        myAvatar->setControllerPoseInSensorFrame(action, pose.transform(avatarToSensorMatrix));
-            leftHandFingerPoses[i] = {
-                userInputMapper->getPoseState((controller::Action)i).transform(avatarToSensorMatrix),
-                userInputMapper->getActionName((controller::Action)i)
-            };
-        }
     }
-    MyAvatar::FingerPosesMap rightHandFingerPoses;
-    if (myAvatar->getRightHandFingerControllerPosesInSensorFrame().size() > 0
-        || userInputMapper->getPoseState(controller::Action::RIGHT_HAND_THUMB1).isValid()) {
-        for (int i = (int)controller::Action::RIGHT_HAND_THUMB1; i <= (int)controller::Action::RIGHT_HAND_PINKY4; i++) {
-            rightHandFingerPoses[i] = {
-                userInputMapper->getPoseState((controller::Action)i).transform(avatarToSensorMatrix),
-                userInputMapper->getActionName((controller::Action)i)
-            };
-        }
-    }
-    myAvatar->setFingerControllerPosesInSensorFrame(leftHandFingerPoses, rightHandFingerPoses);
-
-    controller::Pose leftFootPose = userInputMapper->getPoseState(controller::Action::LEFT_FOOT);
-    controller::Pose rightFootPose = userInputMapper->getPoseState(controller::Action::RIGHT_FOOT);
-    myAvatar->setFootControllerPosesInSensorFrame(leftFootPose.transform(avatarToSensorMatrix), rightFootPose.transform(avatarToSensorMatrix));
-
-    controller::Pose hipsPose = userInputMapper->getPoseState(controller::Action::HIPS);
-    controller::Pose spine2Pose = userInputMapper->getPoseState(controller::Action::SPINE2);
-    myAvatar->setSpineControllerPosesInSensorFrame(hipsPose.transform(avatarToSensorMatrix), spine2Pose.transform(avatarToSensorMatrix));
-
-    controller::Pose headPose = userInputMapper->getPoseState(controller::Action::HEAD);
-    myAvatar->setHeadControllerPoseInSensorFrame(headPose.transform(avatarToSensorMatrix));
-
-    controller::Pose leftArmPose = userInputMapper->getPoseState(controller::Action::LEFT_ARM);
-    controller::Pose rightArmPose = userInputMapper->getPoseState(controller::Action::RIGHT_ARM);
-    myAvatar->setArmControllerPosesInSensorFrame(leftArmPose.transform(avatarToSensorMatrix), rightArmPose.transform(avatarToSensorMatrix));
 
     updateThreads(deltaTime); // If running non-threaded, then give the threads some time to process...
     updateDialogs(deltaTime); // update various stats dialogs if present
@@ -5374,6 +5459,17 @@ void Application::displaySide(RenderArgs* renderArgs, Camera& theCamera, bool se
             }
             renderArgs->_debugFlags = renderDebugFlags;
             //ViveControllerManager::getInstance().updateRendering(renderArgs, _main3DScene, transaction);
+
+            RenderArgs::OutlineFlags renderOutlineFlags = RenderArgs::RENDER_OUTLINE_NONE;
+            auto contextOverlayInterface = DependencyManager::get<ContextOverlayInterface>();
+            if (contextOverlayInterface->getEnabled()) {
+                if (DependencyManager::get<ContextOverlayInterface>()->getIsInMarketplaceInspectionMode()) {
+                    renderOutlineFlags = RenderArgs::RENDER_OUTLINE_MARKETPLACE_MODE;
+                } else {
+                    renderOutlineFlags = RenderArgs::RENDER_OUTLINE_WIREFRAMES;
+                }
+            }
+            renderArgs->_outlineFlags = renderOutlineFlags;
         }
     }
 
@@ -5837,7 +5933,7 @@ void Application::registerScriptEngineWithApplicationServices(ScriptEngine* scri
     auto entityScriptServerLog = DependencyManager::get<EntityScriptServerLogClient>();
     scriptEngine->registerGlobalObject("EntityScriptServerLog", entityScriptServerLog.data());
     scriptEngine->registerGlobalObject("AvatarInputs", AvatarInputs::getInstance());
-    scriptEngine->registerGlobalObject("HoverOverlay", DependencyManager::get<HoverOverlayInterface>().data());
+    scriptEngine->registerGlobalObject("ContextOverlay", DependencyManager::get<ContextOverlayInterface>().data());
 
     qScriptRegisterMetaType(scriptEngine, OverlayIDtoScriptValue, OverlayIDfromScriptValue);
 
@@ -6156,7 +6252,7 @@ void Application::addAssetToWorldFromURLRequestFinished() {
             if (tempFile.open(QIODevice::WriteOnly)) {
                 tempFile.write(request->getData());
                 addAssetToWorldInfoClear(filename);  // Remove message from list; next one added will have a different key.
-                qApp->getFileDownloadInterface()->runUnzip(downloadPath, url, true);
+                qApp->getFileDownloadInterface()->runUnzip(downloadPath, url, true, false);
             } else {
                 QString errorInfo = "Couldn't open temporary file for download";
                 qWarning(interfaceapp) << errorInfo;
@@ -6186,12 +6282,18 @@ void Application::addAssetToWorldUnzipFailure(QString filePath) {
     addAssetToWorldError(filename, "Couldn't unzip file " + filename + ".");
 }
 
-void Application::addAssetToWorld(QString filePath) {
+void Application::addAssetToWorld(QString filePath, QString zipFile, bool isZip) {
     // Automatically upload and add asset to world as an alternative manual process initiated by showAssetServerWidget().
-
+    QString mapping;
-    QString path = QUrl(filePath).toLocalFile();
+    QString path = filePath;
     QString filename = filenameFromPath(path);
-    QString mapping = "/" + filename;
+    if (isZip) {
+        QString assetFolder = zipFile.section("/", -1);
+        assetFolder.remove(".zip");
+        mapping = "/" + assetFolder + "/" + filename;
+    } else {
+        mapping = "/" + filename;
+    }
 
     // Test repeated because possibly different code paths.
     if (!DependencyManager::get<NodeList>()->getThisNodeCanWriteAssets()) {
@@ -6272,7 +6374,13 @@ void Application::addAssetToWorldSetMapping(QString filePath, QString mapping, Q
             qWarning(interfaceapp) << "Error downloading model: " + errorInfo;
             addAssetToWorldError(filenameFromPath(filePath), errorInfo);
         } else {
-            addAssetToWorldAddEntity(filePath, mapping);
+            // to prevent files that aren't models from being loaded into world automatically
+            if (filePath.endsWith(".obj") || filePath.endsWith(".fbx")) {
+                addAssetToWorldAddEntity(filePath, mapping);
+            } else {
+                qCDebug(interfaceapp) << "Zipped contents are not supported entity files";
+                addAssetToWorldInfoDone(filenameFromPath(filePath));
+            }
         }
         request->deleteLater();
     });
@@ -6562,15 +6670,18 @@ void Application::onAssetToWorldMessageBoxClosed() {
 }
 
 
-void Application::handleUnzip(QString zipFile, QString unzipFile, bool autoAdd) {
+void Application::handleUnzip(QString zipFile, QStringList unzipFile, bool autoAdd, bool isZip) {
     if (autoAdd) {
         if (!unzipFile.isEmpty()) {
-            addAssetToWorld(unzipFile);
+            for (int i = 0; i < unzipFile.length(); i++) {
+                qCDebug(interfaceapp) << "Preparing file for asset server: " << unzipFile.at(i);
+                addAssetToWorld(unzipFile.at(i), zipFile, isZip);
+            }
         } else {
             addAssetToWorldUnzipFailure(zipFile);
         }
     } else {
-        showAssetServerWidget(unzipFile);
+        showAssetServerWidget(unzipFile.first());
     }
 }
 
@@ -7033,6 +7144,7 @@ void Application::updateDisplayMode() {
 
         auto oldDisplayPlugin = _displayPlugin;
         if (_displayPlugin) {
+            disconnect(_displayPluginPresentConnection);
             _displayPlugin->deactivate();
         }
 
@@ -7073,6 +7185,7 @@ void Application::updateDisplayMode() {
         _offscreenContext->makeCurrent();
         getApplicationCompositor().setDisplayPlugin(newDisplayPlugin);
         _displayPlugin = newDisplayPlugin;
+        _displayPluginPresentConnection = connect(_displayPlugin.get(), &DisplayPlugin::presented, this, &Application::onPresent);
         offscreenUi->getDesktop()->setProperty("repositionLocked", wasRepositionLocked);
     }
 

interface/src/Application.h

@@ -55,6 +55,7 @@
 #include "BandwidthRecorder.h"
 #include "FancyCamera.h"
 #include "ConnectionMonitor.h"
+#include "CursorManager.h"
 #include "gpu/Context.h"
 #include "Menu.h"
 #include "octree/OctreePacketProcessor.h"
@@ -128,8 +129,7 @@ public:
     virtual DisplayPluginPointer getActiveDisplayPlugin() const override;
 
     enum Event {
-        Present = DisplayPlugin::Present,
+        Paint = QEvent::User + 1,
-        Paint,
         Idle,
         Lambda
     };
@@ -165,7 +165,7 @@ public:
     QSize getDeviceSize() const;
     bool hasFocus() const;
 
-    void showCursor(const QCursor& cursor);
+    void showCursor(const Cursor::Icon& cursor);
 
     bool isThrottleRendering() const;
 
@@ -331,14 +331,14 @@ public slots:
     // FIXME: Move addAssetToWorld* methods to own class?
     void addAssetToWorldFromURL(QString url);
     void addAssetToWorldFromURLRequestFinished();
-    void addAssetToWorld(QString filePath);
+    void addAssetToWorld(QString filePath, QString zipFile, bool isZip);
     void addAssetToWorldUnzipFailure(QString filePath);
     void addAssetToWorldWithNewMapping(QString filePath, QString mapping, int copy);
     void addAssetToWorldUpload(QString filePath, QString mapping);
     void addAssetToWorldSetMapping(QString filePath, QString mapping, QString hash);
     void addAssetToWorldAddEntity(QString filePath, QString mapping);
 
-    void handleUnzip(QString sourceFile, QString destinationFile, bool autoAdd);
+    void handleUnzip(QString sourceFile, QStringList destinationFile, bool autoAdd, bool isZip);
 
     FileScriptingInterface* getFileDownloadInterface() { return _fileDownload; }
 
@@ -400,11 +400,15 @@ public slots:
     void loadDomainConnectionDialog();
     void showScriptLogs();
 
+    const QString getPreferredCursor() const { return _preferredCursor.get(); }
+    void setPreferredCursor(const QString& cursor);
+
 private slots:
     void showDesktop();
     void clearDomainOctreeDetails();
     void clearDomainAvatars();
     void onAboutToQuit();
+    void onPresent(quint32 frameCount);
 
     void resettingDomain();
 
@@ -451,8 +455,8 @@ private:
 
     void cleanupBeforeQuit();
 
-    bool shouldPaint(float nsecsElapsed);
+    bool shouldPaint();
-    void idle(float nsecsElapsed);
+    void idle();
     void update(float deltaTime);
 
     // Various helper functions called during update()
@@ -477,6 +481,7 @@ private:
 
     bool importJSONFromURL(const QString& urlString);
     bool importSVOFromURL(const QString& urlString);
+    bool importFromZIP(const QString& filePath);
 
     bool nearbyEntitiesAreReadyForPhysics();
     int processOctreeStats(ReceivedMessage& message, SharedNodePointer sendingNode);
@@ -514,6 +519,7 @@ private:
 
     OffscreenGLCanvas* _offscreenContext { nullptr };
     DisplayPluginPointer _displayPlugin;
+    QMetaObject::Connection _displayPluginPresentConnection;
     mutable std::mutex _displayPluginLock;
     InputPluginList _activeInputPlugins;
 
@@ -564,6 +570,7 @@ private:
     Setting::Handle<bool> _hmdTabletBecomesToolbarSetting;
     Setting::Handle<bool> _preferAvatarFingerOverStylusSetting;
     Setting::Handle<bool> _constrainToolbarPosition;
+    Setting::Handle<QString> _preferredCursor;
 
     float _scaleMirror;
     float _rotateMirror;
@@ -637,7 +644,7 @@ private:
 
     void checkChangeCursor();
     mutable QMutex _changeCursorLock { QMutex::Recursive };
-    QCursor _desiredCursor{ Qt::BlankCursor };
+    Qt::CursorShape _desiredCursor{ Qt::BlankCursor };
     bool _cursorNeedsChanging { false };
 
     QThread* _deadlockWatchdogThread;

interface/src/Menu.cpp

@@ -222,19 +222,19 @@ Menu::Menu() {
     cameraModeGroup->setExclusive(true);
 
     // View > First Person
-    cameraModeGroup->addAction(addCheckableActionToQMenuAndActionHash(viewMenu,
+    cameraModeGroup->addAction(addCheckableActionToQMenuAndActionHash(
-        MenuOption::FirstPerson, 0,
+                                   viewMenu, MenuOption::FirstPerson, Qt::CTRL | Qt::Key_F,
-        true, qApp, SLOT(cameraMenuChanged())));
+                                   true, qApp, SLOT(cameraMenuChanged())));
 
     // View > Third Person
-    cameraModeGroup->addAction(addCheckableActionToQMenuAndActionHash(viewMenu,
+    cameraModeGroup->addAction(addCheckableActionToQMenuAndActionHash(
-        MenuOption::ThirdPerson, 0,
+                                   viewMenu, MenuOption::ThirdPerson, Qt::CTRL | Qt::Key_G,
-        false, qApp, SLOT(cameraMenuChanged())));
+                                   false, qApp, SLOT(cameraMenuChanged())));
 
     // View > Mirror
-    cameraModeGroup->addAction(addCheckableActionToQMenuAndActionHash(viewMenu,
+    cameraModeGroup->addAction(addCheckableActionToQMenuAndActionHash(
-        MenuOption::FullscreenMirror, 0,
+                                   viewMenu, MenuOption::FullscreenMirror, Qt::CTRL | Qt::Key_H,
-        false, qApp, SLOT(cameraMenuChanged())));
+                                   false, qApp, SLOT(cameraMenuChanged())));
 
     // View > Independent [advanced]
     cameraModeGroup->addAction(addCheckableActionToQMenuAndActionHash(viewMenu,

interface/src/avatar/AvatarActionHold.cpp

@@ -134,9 +134,9 @@ bool AvatarActionHold::getTarget(float deltaTimeStep, glm::quat& rotation, glm::
             // fetch the hand controller pose
             controller::Pose pose;
             if (isRightHand) {
-                pose = myAvatar->getRightHandControllerPoseInWorldFrame();
+                pose = myAvatar->getControllerPoseInWorldFrame(controller::Action::RIGHT_HAND);
             } else {
-                pose = myAvatar->getLeftHandControllerPoseInWorldFrame();
+                pose = myAvatar->getControllerPoseInWorldFrame(controller::Action::LEFT_HAND);
             }
 
             if (pose.isValid()) {

interface/src/avatar/MyAvatar.cpp

@@ -255,6 +255,13 @@ MyAvatar::~MyAvatar() {
     _lookAtTargetAvatar.reset();
 }
 
+void MyAvatar::setDominantHand(const QString& hand) {
+    if (hand == DOMINANT_LEFT_HAND || hand == DOMINANT_RIGHT_HAND) {
+        _dominantHand = hand;
+        emit dominantHandChanged(_dominantHand);
+    }
+}
+
 void MyAvatar::registerMetaTypes(QScriptEngine* engine) {
     QScriptValue value = engine->newQObject(this, QScriptEngine::QtOwnership, QScriptEngine::ExcludeDeleteLater | QScriptEngine::ExcludeChildObjects);
     engine->globalObject().setProperty("MyAvatar", value);
@@ -422,7 +429,7 @@ void MyAvatar::update(float deltaTime) {
     }
 
 #ifdef DEBUG_DRAW_HMD_MOVING_AVERAGE
-    glm::vec3 p = transformPoint(getSensorToWorldMatrix(), getHeadControllerPoseInAvatarFrame() *
+    glm::vec3 p = transformPoint(getSensorToWorldMatrix(), getControllerPoseInAvatarFrame(controller::Pose::HEAD) *
                                  glm::vec3(_headControllerFacingMovingAverage.x, 0.0f, _headControllerFacingMovingAverage.y));
     DebugDraw::getInstance().addMarker("facing-avg", getOrientation(), p, glm::vec4(1.0f));
     p = transformPoint(getSensorToWorldMatrix(), getHMDSensorPosition() +
@@ -607,7 +614,7 @@ void MyAvatar::simulate(float deltaTime) {
             MovingEntitiesOperator moveOperator(entityTree);
             forEachDescendant([&](SpatiallyNestablePointer object) {
                 // if the queryBox has changed, tell the entity-server
-                if (object->computePuffedQueryAACube() && object->getNestableType() == NestableType::Entity) {
+                if (object->getNestableType() == NestableType::Entity && object->checkAndMaybeUpdateQueryAACube()) {
                     EntityItemPointer entity = std::static_pointer_cast<EntityItem>(object);
                     bool success;
                     AACube newCube = entity->getQueryAACube(success);
@@ -657,7 +664,7 @@ void MyAvatar::updateFromHMDSensorMatrix(const glm::mat4& hmdSensorMatrix) {
 
     _hmdSensorPosition = newHmdSensorPosition;
     _hmdSensorOrientation = glm::quat_cast(hmdSensorMatrix);
-    auto headPose = _headControllerPoseInSensorFrameCache.get();
+    auto headPose = getControllerPoseInSensorFrame(controller::Action::HEAD);
     if (headPose.isValid()) {
         _headControllerFacing = getFacingDir2D(headPose.rotation);
     } else {
@@ -753,37 +760,37 @@ void MyAvatar::updateFromTrackers(float deltaTime) {
 }
 
 glm::vec3 MyAvatar::getLeftHandPosition() const {
-    auto pose = getLeftHandControllerPoseInAvatarFrame();
+    auto pose = getControllerPoseInAvatarFrame(controller::Action::LEFT_HAND);
     return pose.isValid() ? pose.getTranslation() : glm::vec3(0.0f);
 }
 
 glm::vec3 MyAvatar::getRightHandPosition() const {
-    auto pose = getRightHandControllerPoseInAvatarFrame();
+    auto pose = getControllerPoseInAvatarFrame(controller::Action::RIGHT_HAND);
     return pose.isValid() ? pose.getTranslation() : glm::vec3(0.0f);
 }
 
 glm::vec3 MyAvatar::getLeftHandTipPosition() const {
     const float TIP_LENGTH = 0.3f;
-    auto pose = getLeftHandControllerPoseInAvatarFrame();
+    auto pose = getControllerPoseInAvatarFrame(controller::Action::LEFT_HAND);
     return pose.isValid() ? pose.getTranslation() * pose.getRotation() + glm::vec3(0.0f, TIP_LENGTH, 0.0f) : glm::vec3(0.0f);
 }
 
 glm::vec3 MyAvatar::getRightHandTipPosition() const {
     const float TIP_LENGTH = 0.3f;
-    auto pose = getRightHandControllerPoseInAvatarFrame();
+    auto pose = getControllerPoseInAvatarFrame(controller::Action::RIGHT_HAND);
     return pose.isValid() ? pose.getTranslation() * pose.getRotation() + glm::vec3(0.0f, TIP_LENGTH, 0.0f) : glm::vec3(0.0f);
 }
 
 controller::Pose MyAvatar::getLeftHandPose() const {
-    return getLeftHandControllerPoseInAvatarFrame();
+    return getControllerPoseInAvatarFrame(controller::Action::LEFT_HAND);
 }
 
 controller::Pose MyAvatar::getRightHandPose() const {
-    return getRightHandControllerPoseInAvatarFrame();
+    return getControllerPoseInAvatarFrame(controller::Action::RIGHT_HAND);
 }
 
 controller::Pose MyAvatar::getLeftHandTipPose() const {
-    auto pose = getLeftHandControllerPoseInAvatarFrame();
+    auto pose = getLeftHandPose();
     glm::vec3 tipTrans = getLeftHandTipPosition();
     pose.velocity += glm::cross(pose.getAngularVelocity(), pose.getTranslation() - tipTrans);
     pose.translation = tipTrans;
@@ -791,7 +798,7 @@ controller::Pose MyAvatar::getLeftHandTipPose() const {
 }
 
 controller::Pose MyAvatar::getRightHandTipPose() const {
-    auto pose = getRightHandControllerPoseInAvatarFrame();
+    auto pose = getRightHandPose();
     glm::vec3 tipTrans = getRightHandTipPosition();
     pose.velocity += glm::cross(pose.getAngularVelocity(), pose.getTranslation() - tipTrans);
     pose.translation = tipTrans;
@@ -926,6 +933,7 @@ void MyAvatar::saveData() {
     Settings settings;
     settings.beginGroup("Avatar");
 
+    settings.setValue("dominantHand", _dominantHand);
     settings.setValue("headPitch", getHead()->getBasePitch());
 
     settings.setValue("scale", _targetScale);
@@ -1122,7 +1130,7 @@ void MyAvatar::loadData() {
     setCollisionSoundURL(settings.value("collisionSoundURL", DEFAULT_AVATAR_COLLISION_SOUND_URL).toString());
     setSnapTurn(settings.value("useSnapTurn", _useSnapTurn).toBool());
     setClearOverlayWhenMoving(settings.value("clearOverlayWhenMoving", _clearOverlayWhenMoving).toBool());
-
+    setDominantHand(settings.value("dominantHand", _dominantHand).toString().toLower());
     settings.endGroup();
 
     setEnableMeshVisible(Menu::getInstance()->isOptionChecked(MenuOption::MeshVisible));
@@ -1291,7 +1299,7 @@ eyeContactTarget MyAvatar::getEyeContactTarget() {
 }
 
 glm::vec3 MyAvatar::getDefaultEyePosition() const {
-    return getPosition() + getWorldAlignedOrientation() * Quaternions::Y_180 * _skeletonModel->getDefaultEyeModelPosition();
+    return getPosition() + getOrientation() * Quaternions::Y_180 * _skeletonModel->getDefaultEyeModelPosition();
 }
 
 const float SCRIPT_PRIORITY = 1.0f + 1.0f;
@@ -1422,159 +1430,43 @@ void MyAvatar::rebuildCollisionShape() {
     _characterController.setLocalBoundingBox(corner, diagonal);
 }
 
-
+void MyAvatar::setControllerPoseInSensorFrame(controller::Action action, const controller::Pose& pose) {
-void MyAvatar::setHandControllerPosesInSensorFrame(const controller::Pose& left, const controller::Pose& right) {
+    std::lock_guard<std::mutex> guard(_controllerPoseMapMutex);
-    _leftHandControllerPoseInSensorFrameCache.set(left);
+    auto iter = _controllerPoseMap.find(action);
-    _rightHandControllerPoseInSensorFrameCache.set(right);
+    if (iter != _controllerPoseMap.end()) {
+        iter->second = pose;
+    } else {
+        _controllerPoseMap.insert({ action, pose });
+    }
 }
 
-controller::Pose MyAvatar::getLeftHandControllerPoseInSensorFrame() const {
+controller::Pose MyAvatar::getControllerPoseInSensorFrame(controller::Action action) const {
-    return _leftHandControllerPoseInSensorFrameCache.get();
+    std::lock_guard<std::mutex> guard(_controllerPoseMapMutex);
+    auto iter = _controllerPoseMap.find(action);
+    if (iter != _controllerPoseMap.end()) {
+        return iter->second;
+    } else {
+        return controller::Pose(); // invalid pose
+    }
 }
 
-controller::Pose MyAvatar::getRightHandControllerPoseInSensorFrame() const {
+controller::Pose MyAvatar::getControllerPoseInWorldFrame(controller::Action action) const {
-    return _rightHandControllerPoseInSensorFrameCache.get();
+    auto pose = getControllerPoseInSensorFrame(action);
+    if (pose.valid) {
+        return pose.transform(getSensorToWorldMatrix());
+    } else {
+        return controller::Pose(); // invalid pose
+    }
 }
 
-controller::Pose MyAvatar::getLeftHandControllerPoseInWorldFrame() const {
+controller::Pose MyAvatar::getControllerPoseInAvatarFrame(controller::Action action) const {
-    return _leftHandControllerPoseInSensorFrameCache.get().transform(getSensorToWorldMatrix());
+    auto pose = getControllerPoseInWorldFrame(action);
-}
+    if (pose.valid) {
-
+        glm::mat4 invAvatarMatrix = glm::inverse(createMatFromQuatAndPos(getOrientation(), getPosition()));
-controller::Pose MyAvatar::getRightHandControllerPoseInWorldFrame() const {
+        return pose.transform(invAvatarMatrix);
-    return _rightHandControllerPoseInSensorFrameCache.get().transform(getSensorToWorldMatrix());
+    } else {
-}
+        return controller::Pose(); // invalid pose
-
+    }
-controller::Pose MyAvatar::getLeftHandControllerPoseInAvatarFrame() const {
-    glm::mat4 invAvatarMatrix = glm::inverse(createMatFromQuatAndPos(getOrientation(), getPosition()));
-    return getLeftHandControllerPoseInWorldFrame().transform(invAvatarMatrix);
-}
-
-controller::Pose MyAvatar::getRightHandControllerPoseInAvatarFrame() const {
-    glm::mat4 invAvatarMatrix = glm::inverse(createMatFromQuatAndPos(getOrientation(), getPosition()));
-    return getRightHandControllerPoseInWorldFrame().transform(invAvatarMatrix);
-}
-
-void MyAvatar::setFingerControllerPosesInSensorFrame(const FingerPosesMap& left, const FingerPosesMap& right) {
-    _leftHandFingerPosesInSensorFramceCache.set(left);
-    _rightHandFingerPosesInSensorFramceCache.set(right);
-}
-
-MyAvatar::FingerPosesMap MyAvatar::getLeftHandFingerControllerPosesInSensorFrame() const {
-    return _leftHandFingerPosesInSensorFramceCache.get();
-}
-
-MyAvatar::FingerPosesMap MyAvatar::getRightHandFingerControllerPosesInSensorFrame() const {
-    return _rightHandFingerPosesInSensorFramceCache.get();
-}
-
-void MyAvatar::setFootControllerPosesInSensorFrame(const controller::Pose& left, const controller::Pose& right) {
-    _leftFootControllerPoseInSensorFrameCache.set(left);
-    _rightFootControllerPoseInSensorFrameCache.set(right);
-}
-
-controller::Pose MyAvatar::getLeftFootControllerPoseInSensorFrame() const {
-    return _leftFootControllerPoseInSensorFrameCache.get();
-}
-
-controller::Pose MyAvatar::getRightFootControllerPoseInSensorFrame() const {
-    return _rightFootControllerPoseInSensorFrameCache.get();
-}
-
-controller::Pose MyAvatar::getLeftFootControllerPoseInWorldFrame() const {
-    return _leftFootControllerPoseInSensorFrameCache.get().transform(getSensorToWorldMatrix());
-}
-
-controller::Pose MyAvatar::getRightFootControllerPoseInWorldFrame() const {
-    return _rightFootControllerPoseInSensorFrameCache.get().transform(getSensorToWorldMatrix());
-}
-
-controller::Pose MyAvatar::getLeftFootControllerPoseInAvatarFrame() const {
-    glm::mat4 invAvatarMatrix = glm::inverse(createMatFromQuatAndPos(getOrientation(), getPosition()));
-    return getLeftFootControllerPoseInWorldFrame().transform(invAvatarMatrix);
-}
-
-controller::Pose MyAvatar::getRightFootControllerPoseInAvatarFrame() const {
-    glm::mat4 invAvatarMatrix = glm::inverse(createMatFromQuatAndPos(getOrientation(), getPosition()));
-    return getRightFootControllerPoseInWorldFrame().transform(invAvatarMatrix);
-}
-
-void MyAvatar::setSpineControllerPosesInSensorFrame(const controller::Pose& hips, const controller::Pose& spine2) {
-    _hipsControllerPoseInSensorFrameCache.set(hips);
-    _spine2ControllerPoseInSensorFrameCache.set(spine2);
-}
-
-controller::Pose MyAvatar::getHipsControllerPoseInSensorFrame() const {
-    return _hipsControllerPoseInSensorFrameCache.get();
-}
-
-controller::Pose MyAvatar::getSpine2ControllerPoseInSensorFrame() const {
-    return _spine2ControllerPoseInSensorFrameCache.get();
-}
-
-controller::Pose MyAvatar::getHipsControllerPoseInWorldFrame() const {
-    return _hipsControllerPoseInSensorFrameCache.get().transform(getSensorToWorldMatrix());
-}
-
-controller::Pose MyAvatar::getSpine2ControllerPoseInWorldFrame() const {
-    return _spine2ControllerPoseInSensorFrameCache.get().transform(getSensorToWorldMatrix());
-}
-
-controller::Pose MyAvatar::getHipsControllerPoseInAvatarFrame() const {
-    glm::mat4 invAvatarMatrix = glm::inverse(createMatFromQuatAndPos(getOrientation(), getPosition()));
-    return getHipsControllerPoseInWorldFrame().transform(invAvatarMatrix);
-}
-
-controller::Pose MyAvatar::getSpine2ControllerPoseInAvatarFrame() const {
-    glm::mat4 invAvatarMatrix = glm::inverse(createMatFromQuatAndPos(getOrientation(), getPosition()));
-    return getSpine2ControllerPoseInWorldFrame().transform(invAvatarMatrix);
-}
-
-void MyAvatar::setHeadControllerPoseInSensorFrame(const controller::Pose& head) {
-    _headControllerPoseInSensorFrameCache.set(head);
-}
-
-controller::Pose MyAvatar::getHeadControllerPoseInSensorFrame() const {
-    return _headControllerPoseInSensorFrameCache.get();
-}
-
-controller::Pose MyAvatar::getHeadControllerPoseInWorldFrame() const {
-    return _headControllerPoseInSensorFrameCache.get().transform(getSensorToWorldMatrix());
-}
-
-controller::Pose MyAvatar::getHeadControllerPoseInAvatarFrame() const {
-    glm::mat4 invAvatarMatrix = glm::inverse(createMatFromQuatAndPos(getOrientation(), getPosition()));
-    return getHeadControllerPoseInWorldFrame().transform(invAvatarMatrix);
-}
-
-void MyAvatar::setArmControllerPosesInSensorFrame(const controller::Pose& left, const controller::Pose& right) {
-    _leftArmControllerPoseInSensorFrameCache.set(left);
-    _rightArmControllerPoseInSensorFrameCache.set(right);
-}
-
-controller::Pose MyAvatar::getLeftArmControllerPoseInSensorFrame() const {
-    return _leftArmControllerPoseInSensorFrameCache.get();
-}
-
-controller::Pose MyAvatar::getRightArmControllerPoseInSensorFrame() const {
-    return _rightArmControllerPoseInSensorFrameCache.get();
-}
-
-controller::Pose MyAvatar::getLeftArmControllerPoseInWorldFrame() const {
-    return getLeftArmControllerPoseInSensorFrame().transform(getSensorToWorldMatrix());
-}
-
-controller::Pose MyAvatar::getRightArmControllerPoseInWorldFrame() const {
-    return getRightArmControllerPoseInSensorFrame().transform(getSensorToWorldMatrix());
-}
-
-controller::Pose MyAvatar::getLeftArmControllerPoseInAvatarFrame() const {
-    glm::mat4 worldToAvatarMat = glm::inverse(createMatFromQuatAndPos(getOrientation(), getPosition()));
-    return getLeftArmControllerPoseInWorldFrame().transform(worldToAvatarMat);
-}
-
-controller::Pose MyAvatar::getRightArmControllerPoseInAvatarFrame() const {
-    glm::mat4 worldToAvatarMat = glm::inverse(createMatFromQuatAndPos(getOrientation(), getPosition()));
-    return getRightArmControllerPoseInWorldFrame().transform(worldToAvatarMat);
 }
 
 void MyAvatar::updateMotors() {
@@ -1586,9 +1478,14 @@ void MyAvatar::updateMotors() {
             motorRotation = getMyHead()->getHeadOrientation();
         } else {
             // non-hovering = walking: follow camera twist about vertical but not lift
-            // so we decompose camera's rotation and store the twist part in motorRotation
+            // we decompose camera's rotation and store the twist part in motorRotation
+            // however, we need to perform the decomposition in the avatar-frame
+            // using the local UP axis and then transform back into world-frame
+            glm::quat orientation = getOrientation();
+            glm::quat headOrientation = glm::inverse(orientation) * getMyHead()->getHeadOrientation(); // avatar-frame
             glm::quat liftRotation;
-            swingTwistDecomposition(getMyHead()->getHeadOrientation(), _worldUpDirection, liftRotation, motorRotation);
+            swingTwistDecomposition(headOrientation, Vectors::UNIT_Y, liftRotation, motorRotation);
+            motorRotation = orientation * motorRotation;
         }
         const float DEFAULT_MOTOR_TIMESCALE = 0.2f;
         const float INVALID_MOTOR_TIMESCALE = 1.0e6f;
@@ -1632,7 +1529,7 @@ void MyAvatar::prepareForPhysicsSimulation() {
     _characterController.setParentVelocity(parentVelocity);
 
     _characterController.setPositionAndOrientation(getPosition(), getOrientation());
-    auto headPose = getHeadControllerPoseInAvatarFrame();
+    auto headPose = getControllerPoseInAvatarFrame(controller::Action::HEAD);
     if (headPose.isValid()) {
         _follow.prePhysicsUpdate(*this, deriveBodyFromHMDSensor(), _bodySensorMatrix, hasDriveInput());
     } else {
@@ -1642,11 +1539,31 @@ void MyAvatar::prepareForPhysicsSimulation() {
     _prePhysicsRoomPose = AnimPose(_sensorToWorldMatrix);
 }
 
+// There are a number of possible strategies for this set of tools through endRender, below.
+void MyAvatar::nextAttitude(glm::vec3 position, glm::quat orientation) {
+    bool success;
+    Transform trans = getTransform(success);
+    if (!success) {
+        qCWarning(interfaceapp) << "Warning -- MyAvatar::nextAttitude failed";
+        return;
+    }
+    trans.setTranslation(position);
+    trans.setRotation(orientation);
+    SpatiallyNestable::setTransform(trans, success);
+    if (!success) {
+        qCWarning(interfaceapp) << "Warning -- MyAvatar::nextAttitude failed";
+    }
+    updateAttitude(orientation);
+}
+
 void MyAvatar::harvestResultsFromPhysicsSimulation(float deltaTime) {
-    glm::vec3 position = getPosition();
+    glm::vec3 position;
-    glm::quat orientation = getOrientation();
+    glm::quat orientation;
     if (_characterController.isEnabledAndReady()) {
         _characterController.getPositionAndOrientation(position, orientation);
+    } else {
+        position = getPosition();
+        orientation = getOrientation();
     }
     nextAttitude(position, orientation);
     _bodySensorMatrix = _follow.postPhysicsUpdate(*this, _bodySensorMatrix);
@@ -1866,8 +1783,8 @@ void MyAvatar::postUpdate(float deltaTime) {
     }
 
     if (_enableDebugDrawHandControllers) {
-        auto leftHandPose = getLeftHandControllerPoseInWorldFrame();
+        auto leftHandPose = getControllerPoseInWorldFrame(controller::Action::LEFT_HAND);
-        auto rightHandPose = getRightHandControllerPoseInWorldFrame();
+        auto rightHandPose = getControllerPoseInWorldFrame(controller::Action::RIGHT_HAND);
 
         if (leftHandPose.isValid()) {
             DebugDraw::getInstance().addMarker("leftHandController", leftHandPose.getRotation(), leftHandPose.getTranslation(), glm::vec4(1));
@@ -2020,7 +1937,7 @@ void MyAvatar::updateOrientation(float deltaTime) {
 
     getHead()->setBasePitch(getHead()->getBasePitch() + getDriveKey(PITCH) * _pitchSpeed * deltaTime);
 
-    auto headPose = getHeadControllerPoseInAvatarFrame();
+    auto headPose = getControllerPoseInAvatarFrame(controller::Action::HEAD);
     if (headPose.isValid()) {
         glm::quat localOrientation = headPose.rotation * Quaternions::Y_180;
         // these angles will be in radians
@@ -2656,10 +2573,10 @@ bool MyAvatar::isDriveKeyDisabled(DriveKeys key) const {
 glm::mat4 MyAvatar::deriveBodyFromHMDSensor() const {
     glm::vec3 headPosition;
     glm::quat headOrientation;
-    auto headPose = getHeadControllerPoseInSensorFrame();
+    auto headPose = getControllerPoseInSensorFrame(controller::Action::HEAD);
     if (headPose.isValid()) {
-        headPosition = getHeadControllerPoseInSensorFrame().translation;
+        headPosition = headPose.translation;
-        headOrientation = getHeadControllerPoseInSensorFrame().rotation * Quaternions::Y_180;
+        headOrientation = headPose.rotation * Quaternions::Y_180;
     }
     const glm::quat headOrientationYawOnly = cancelOutRollAndPitch(headOrientation);
 
@@ -2841,7 +2758,8 @@ bool MyAvatar::FollowHelper::shouldActivateVertical(const MyAvatar& myAvatar, co
 void MyAvatar::FollowHelper::prePhysicsUpdate(MyAvatar& myAvatar, const glm::mat4& desiredBodyMatrix,
                                               const glm::mat4& currentBodyMatrix, bool hasDriveInput) {
 
-    if (myAvatar.getHMDLeanRecenterEnabled()) {
+    if (myAvatar.getHMDLeanRecenterEnabled() &&
+        qApp->getCamera().getMode() != CAMERA_MODE_MIRROR) {
         if (!isActive(Rotation) && (shouldActivateRotation(myAvatar, desiredBodyMatrix, currentBodyMatrix) || hasDriveInput)) {
             activate(Rotation);
         }
@@ -2968,19 +2886,19 @@ glm::quat MyAvatar::getAbsoluteJointRotationInObjectFrame(int index) const {
 
     switch (index) {
         case CONTROLLER_LEFTHAND_INDEX: {
-            return getLeftHandControllerPoseInAvatarFrame().getRotation();
+            return getControllerPoseInAvatarFrame(controller::Action::LEFT_HAND).getRotation();
         }
         case CONTROLLER_RIGHTHAND_INDEX: {
-            return getRightHandControllerPoseInAvatarFrame().getRotation();
+            return getControllerPoseInAvatarFrame(controller::Action::RIGHT_HAND).getRotation();
         }
         case CAMERA_RELATIVE_CONTROLLER_LEFTHAND_INDEX: {
-            auto pose = _leftHandControllerPoseInSensorFrameCache.get();
+            auto pose = getControllerPoseInSensorFrame(controller::Action::LEFT_HAND);
             glm::mat4 controllerSensorMatrix = createMatFromQuatAndPos(pose.rotation, pose.translation);
             glm::mat4 result = computeCameraRelativeHandControllerMatrix(controllerSensorMatrix);
             return glmExtractRotation(result);
         }
         case CAMERA_RELATIVE_CONTROLLER_RIGHTHAND_INDEX: {
-            auto pose = _rightHandControllerPoseInSensorFrameCache.get();
+            auto pose = getControllerPoseInSensorFrame(controller::Action::RIGHT_HAND);
             glm::mat4 controllerSensorMatrix = createMatFromQuatAndPos(pose.rotation, pose.translation);
             glm::mat4 result = computeCameraRelativeHandControllerMatrix(controllerSensorMatrix);
             return glmExtractRotation(result);
@@ -3005,19 +2923,19 @@ glm::vec3 MyAvatar::getAbsoluteJointTranslationInObjectFrame(int index) const {
 
     switch (index) {
         case CONTROLLER_LEFTHAND_INDEX: {
-            return getLeftHandControllerPoseInAvatarFrame().getTranslation();
+            return getControllerPoseInAvatarFrame(controller::Action::LEFT_HAND).getTranslation();
         }
         case CONTROLLER_RIGHTHAND_INDEX: {
-            return getRightHandControllerPoseInAvatarFrame().getTranslation();
+            return getControllerPoseInAvatarFrame(controller::Action::RIGHT_HAND).getTranslation();
         }
         case CAMERA_RELATIVE_CONTROLLER_LEFTHAND_INDEX: {
-            auto pose = _leftHandControllerPoseInSensorFrameCache.get();
+            auto pose = getControllerPoseInSensorFrame(controller::Action::LEFT_HAND);
             glm::mat4 controllerSensorMatrix = createMatFromQuatAndPos(pose.rotation, pose.translation);
             glm::mat4 result = computeCameraRelativeHandControllerMatrix(controllerSensorMatrix);
             return extractTranslation(result);
         }
         case CAMERA_RELATIVE_CONTROLLER_RIGHTHAND_INDEX: {
-            auto pose = _rightHandControllerPoseInSensorFrameCache.get();
+            auto pose = getControllerPoseInSensorFrame(controller::Action::RIGHT_HAND);
             glm::mat4 controllerSensorMatrix = createMatFromQuatAndPos(pose.rotation, pose.translation);
             glm::mat4 result = computeCameraRelativeHandControllerMatrix(controllerSensorMatrix);
             return extractTranslation(result);

interface/src/avatar/MyAvatar.h

@@ -43,6 +43,7 @@ enum AudioListenerMode {
     FROM_CAMERA,
     CUSTOM
 };
+
 Q_DECLARE_METATYPE(AudioListenerMode);
 
 class MyAvatar : public Avatar {
@@ -141,6 +142,9 @@ class MyAvatar : public Avatar {
     Q_PROPERTY(float hmdRollControlDeadZone READ getHMDRollControlDeadZone WRITE setHMDRollControlDeadZone)
     Q_PROPERTY(float hmdRollControlRate READ getHMDRollControlRate WRITE setHMDRollControlRate)
 
+    const QString DOMINANT_LEFT_HAND = "left";
+    const QString DOMINANT_RIGHT_HAND = "right";
+
 public:
     enum DriveKeys {
         TRANSLATE_X = 0,
@@ -339,6 +343,9 @@ public:
     Q_INVOKABLE bool getClearOverlayWhenMoving() const { return _clearOverlayWhenMoving; }
     Q_INVOKABLE void setClearOverlayWhenMoving(bool on) { _clearOverlayWhenMoving = on; }
 
+    Q_INVOKABLE void setDominantHand(const QString& hand);
+    Q_INVOKABLE QString getDominantHand() const { return _dominantHand; }
+
     Q_INVOKABLE void setHMDLeanRecenterEnabled(bool value) { _hmdLeanRecenterEnabled = value; }
     Q_INVOKABLE bool getHMDLeanRecenterEnabled() const { return _hmdLeanRecenterEnabled; }
 
@@ -424,7 +431,6 @@ public:
     Q_INVOKABLE QString getFullAvatarModelName() const { return _fullAvatarModelName; }
     void resetFullAvatarURL();
 
-
     virtual void setAttachmentData(const QVector<AttachmentData>& attachmentData) override;
 
     MyCharacterController* getCharacterController() { return &_characterController; }
@@ -432,6 +438,7 @@ public:
 
     void updateMotors();
     void prepareForPhysicsSimulation();
+    void nextAttitude(glm::vec3 position, glm::quat orientation); // Can be safely called at any time.
     void harvestResultsFromPhysicsSimulation(float deltaTime);
 
     const QString& getCollisionSoundURL() { return _collisionSoundURL; }
@@ -466,49 +473,12 @@ public:
 
     virtual void rebuildCollisionShape() override;
 
-    void setHandControllerPosesInSensorFrame(const controller::Pose& left, const controller::Pose& right);
-    controller::Pose getLeftHandControllerPoseInSensorFrame() const;
-    controller::Pose getRightHandControllerPoseInSensorFrame() const;
-    controller::Pose getLeftHandControllerPoseInWorldFrame() const;
-    controller::Pose getRightHandControllerPoseInWorldFrame() const;
-    controller::Pose getLeftHandControllerPoseInAvatarFrame() const;
-    controller::Pose getRightHandControllerPoseInAvatarFrame() const;
-
-    typedef std::map<int, std::pair<controller::Pose, QString>> FingerPosesMap;
-    void setFingerControllerPosesInSensorFrame(const FingerPosesMap& left, const FingerPosesMap& right);
-    FingerPosesMap getLeftHandFingerControllerPosesInSensorFrame() const;
-    FingerPosesMap getRightHandFingerControllerPosesInSensorFrame() const;
-
-    void setFootControllerPosesInSensorFrame(const controller::Pose& left, const controller::Pose& right);
-    controller::Pose getLeftFootControllerPoseInSensorFrame() const;
-    controller::Pose getRightFootControllerPoseInSensorFrame() const;
-    controller::Pose getLeftFootControllerPoseInWorldFrame() const;
-    controller::Pose getRightFootControllerPoseInWorldFrame() const;
-    controller::Pose getLeftFootControllerPoseInAvatarFrame() const;
-    controller::Pose getRightFootControllerPoseInAvatarFrame() const;
-
-    void setSpineControllerPosesInSensorFrame(const controller::Pose& hips, const controller::Pose& spine2);
-    controller::Pose getHipsControllerPoseInSensorFrame() const;
-    controller::Pose getSpine2ControllerPoseInSensorFrame() const;
-    controller::Pose getHipsControllerPoseInWorldFrame() const;
-    controller::Pose getSpine2ControllerPoseInWorldFrame() const;
-    controller::Pose getHipsControllerPoseInAvatarFrame() const;
-    controller::Pose getSpine2ControllerPoseInAvatarFrame() const;
-
-    void setHeadControllerPoseInSensorFrame(const controller::Pose& head);
-    controller::Pose getHeadControllerPoseInSensorFrame() const;
-    controller::Pose getHeadControllerPoseInWorldFrame() const;
-    controller::Pose getHeadControllerPoseInAvatarFrame() const;
     const glm::vec2& getHeadControllerFacingMovingAverage() const { return _headControllerFacingMovingAverage; }
 
-
+    void setControllerPoseInSensorFrame(controller::Action action, const controller::Pose& pose);
-    void setArmControllerPosesInSensorFrame(const controller::Pose& left, const controller::Pose& right);
+    controller::Pose getControllerPoseInSensorFrame(controller::Action action) const;
-    controller::Pose getLeftArmControllerPoseInSensorFrame() const;
+    controller::Pose getControllerPoseInWorldFrame(controller::Action action) const;
-    controller::Pose getRightArmControllerPoseInSensorFrame() const;
+    controller::Pose getControllerPoseInAvatarFrame(controller::Action action) const;
-    controller::Pose getLeftArmControllerPoseInWorldFrame() const;
-    controller::Pose getRightArmControllerPoseInWorldFrame() const;
-    controller::Pose getLeftArmControllerPoseInAvatarFrame() const;
-    controller::Pose getRightArmControllerPoseInAvatarFrame() const;
 
     bool hasDriveInput() const;
 
@@ -551,7 +521,6 @@ public:
     Q_INVOKABLE bool isUp(const glm::vec3& direction) { return glm::dot(direction, _worldUpDirection) > 0.0f; }; // true iff direction points up wrt avatar's definition of up.
     Q_INVOKABLE bool isDown(const glm::vec3& direction) { return glm::dot(direction, _worldUpDirection) < 0.0f; };
 
-
 public slots:
     void increaseSize();
     void decreaseSize();
@@ -609,6 +578,7 @@ signals:
     void wentAway();
     void wentActive();
     void skeletonChanged();
+    void dominantHandChanged(const QString& hand);
 
 private:
 
@@ -720,6 +690,7 @@ private:
     QUrl _fstAnimGraphOverrideUrl;
     bool _useSnapTurn { true };
     bool _clearOverlayWhenMoving { true };
+    QString _dominantHand { DOMINANT_RIGHT_HAND };
 
     const float ROLL_CONTROL_DEAD_ZONE_DEFAULT = 8.0f; // deg
     const float ROLL_CONTROL_RATE_DEFAULT = 2.5f; // deg/sec/deg
@@ -796,18 +767,9 @@ private:
     bool _hoverReferenceCameraFacingIsCaptured { false };
     glm::vec3 _hoverReferenceCameraFacing { 0.0f, 0.0f, -1.0f }; // hmd sensor space
 
-    // These are stored in SENSOR frame
+    // all poses are in sensor-frame
-    ThreadSafeValueCache<controller::Pose> _leftHandControllerPoseInSensorFrameCache { controller::Pose() };
+    std::map<controller::Action, controller::Pose> _controllerPoseMap;
-    ThreadSafeValueCache<controller::Pose> _rightHandControllerPoseInSensorFrameCache { controller::Pose() };
+    mutable std::mutex _controllerPoseMapMutex;
-    ThreadSafeValueCache<FingerPosesMap> _leftHandFingerPosesInSensorFramceCache { };
-    ThreadSafeValueCache<FingerPosesMap> _rightHandFingerPosesInSensorFramceCache { };
-    ThreadSafeValueCache<controller::Pose> _leftFootControllerPoseInSensorFrameCache { controller::Pose() };
-    ThreadSafeValueCache<controller::Pose> _rightFootControllerPoseInSensorFrameCache { controller::Pose() };
-    ThreadSafeValueCache<controller::Pose> _hipsControllerPoseInSensorFrameCache { controller::Pose() };
-    ThreadSafeValueCache<controller::Pose> _spine2ControllerPoseInSensorFrameCache { controller::Pose() };
-    ThreadSafeValueCache<controller::Pose> _headControllerPoseInSensorFrameCache { controller::Pose() };
-    ThreadSafeValueCache<controller::Pose> _leftArmControllerPoseInSensorFrameCache { controller::Pose() };
-    ThreadSafeValueCache<controller::Pose> _rightArmControllerPoseInSensorFrameCache { controller::Pose() };
 
     bool _hmdLeanRecenterEnabled = true;
     AnimPose _prePhysicsRoomPose;

interface/src/avatar/MyCharacterController.cpp

@@ -36,7 +36,7 @@ MyCharacterController::~MyCharacterController() {
 
 void MyCharacterController::setDynamicsWorld(btDynamicsWorld* world) {
     CharacterController::setDynamicsWorld(world);
-    if (world) {
+    if (world && _rigidBody) {
         initRayShotgun(world);
     }
 }

interface/src/avatar/MyHead.cpp

@@ -29,17 +29,17 @@ MyHead::MyHead(MyAvatar* owningAvatar) : Head(owningAvatar) {
 glm::quat MyHead::getHeadOrientation() const {
     // NOTE: Head::getHeadOrientation() is not used for orienting the camera "view" while in Oculus mode, so
     // you may wonder why this code is here. This method will be called while in Oculus mode to determine how
-    // to change the driving direction while in Oculus mode. It is used to support driving toward where you're
+    // to change the driving direction while in Oculus mode. It is used to support driving toward where your
     // head is looking. Note that in oculus mode, your actual camera view and where your head is looking is not
     // always the same.
 
     MyAvatar* myAvatar = static_cast<MyAvatar*>(_owningAvatar);
-    auto headPose = myAvatar->getHeadControllerPoseInWorldFrame();
+    auto headPose = myAvatar->getControllerPoseInWorldFrame(controller::Action::HEAD);
     if (headPose.isValid()) {
         return headPose.rotation * Quaternions::Y_180;
     }
 
-    return myAvatar->getWorldAlignedOrientation() * glm::quat(glm::radians(glm::vec3(_basePitch, 0.0f, 0.0f)));
+    return myAvatar->getOrientation() * glm::quat(glm::radians(glm::vec3(_basePitch, 0.0f, 0.0f)));
 }
 
 void MyHead::simulate(float deltaTime) {

interface/src/avatar/MySkeletonModel.cpp

@@ -46,105 +46,82 @@ void MySkeletonModel::updateRig(float deltaTime, glm::mat4 parentTransform) {
     }
 
     MyAvatar* myAvatar = static_cast<MyAvatar*>(_owningAvatar);
+    assert(myAvatar);
 
     Rig::ControllerParameters params;
 
     AnimPose avatarToRigPose(glm::vec3(1.0f), Quaternions::Y_180, glm::vec3(0.0f));
 
     // input action is the highest priority source for head orientation.
-    auto avatarHeadPose = myAvatar->getHeadControllerPoseInAvatarFrame();
+    auto avatarHeadPose = myAvatar->getControllerPoseInAvatarFrame(controller::Action::HEAD);
     if (avatarHeadPose.isValid()) {
         AnimPose pose(avatarHeadPose.getRotation(), avatarHeadPose.getTranslation());
-        params.controllerPoses[Rig::ControllerType_Head] = avatarToRigPose * pose;
+        params.primaryControllerPoses[Rig::PrimaryControllerType_Head] = avatarToRigPose * pose;
-        params.controllerActiveFlags[Rig::ControllerType_Head] = true;
+        params.primaryControllerActiveFlags[Rig::PrimaryControllerType_Head] = true;
     } else {
         // even though full head IK is disabled, the rig still needs the head orientation to rotate the head up and
         // down in desktop mode.
         // preMult 180 is necessary to convert from avatar to rig coordinates.
         // postMult 180 is necessary to convert head from -z forward to z forward.
         glm::quat headRot = Quaternions::Y_180 * head->getFinalOrientationInLocalFrame() * Quaternions::Y_180;
-        params.controllerPoses[Rig::ControllerType_Head] = AnimPose(glm::vec3(1.0f), headRot, glm::vec3(0.0f));
+        params.primaryControllerPoses[Rig::PrimaryControllerType_Head] = AnimPose(glm::vec3(1.0f), headRot, glm::vec3(0.0f));
-        params.controllerActiveFlags[Rig::ControllerType_Head] = false;
+        params.primaryControllerActiveFlags[Rig::PrimaryControllerType_Head] = false;
     }
 
-    auto avatarHipsPose = myAvatar->getHipsControllerPoseInAvatarFrame();
+    //
-    if (avatarHipsPose.isValid()) {
+    // primary controller poses, control IK targets directly.
-        AnimPose pose(avatarHipsPose.getRotation(), avatarHipsPose.getTranslation());
+    //
-        params.controllerPoses[Rig::ControllerType_Hips] = avatarToRigPose * pose;
+
-        params.controllerActiveFlags[Rig::ControllerType_Hips] = true;
+    static const std::vector<std::pair<controller::Action, Rig::PrimaryControllerType>> primaryControllers = {
-    } else {
+        { controller::Action::LEFT_HAND, Rig::PrimaryControllerType_LeftHand },
-        params.controllerPoses[Rig::ControllerType_Hips] = AnimPose::identity;
+        { controller::Action::RIGHT_HAND, Rig::PrimaryControllerType_RightHand },
-        params.controllerActiveFlags[Rig::ControllerType_Hips] = false;
+        { controller::Action::HIPS, Rig::PrimaryControllerType_Hips },
+        { controller::Action::LEFT_FOOT, Rig::PrimaryControllerType_LeftFoot },
+        { controller::Action::RIGHT_FOOT, Rig::PrimaryControllerType_RightFoot },
+        { controller::Action::SPINE2, Rig::PrimaryControllerType_Spine2 }
+    };
+
+    for (auto pair : primaryControllers) {
+        auto controllerPose = myAvatar->getControllerPoseInAvatarFrame(pair.first);
+        if (controllerPose.isValid()) {
+            AnimPose pose(controllerPose.getRotation(), controllerPose.getTranslation());
+            params.primaryControllerPoses[pair.second] = avatarToRigPose * pose;
+            params.primaryControllerActiveFlags[pair.second] = true;
+        } else {
+            params.primaryControllerPoses[pair.second] = AnimPose::identity;
+            params.primaryControllerActiveFlags[pair.second] = false;
+        }
     }
 
-    auto avatarSpine2Pose = myAvatar->getSpine2ControllerPoseInAvatarFrame();
+    //
-    if (avatarSpine2Pose.isValid()) {
+    // secondary controller poses, influence the pose of the skeleton indirectly.
-        AnimPose pose(avatarSpine2Pose.getRotation(), avatarSpine2Pose.getTranslation());
+    //
-        params.controllerPoses[Rig::ControllerType_Spine2] = avatarToRigPose * pose;
-        params.controllerActiveFlags[Rig::ControllerType_Spine2] = true;
-    } else {
-        params.controllerPoses[Rig::ControllerType_Spine2] = AnimPose::identity;
-        params.controllerActiveFlags[Rig::ControllerType_Spine2] = false;
-    }
 
-    auto avatarRightArmPose = myAvatar->getRightArmControllerPoseInAvatarFrame();
+    static const std::vector<std::pair<controller::Action, Rig::SecondaryControllerType>> secondaryControllers = {
-    if (avatarRightArmPose.isValid()) {
+        { controller::Action::LEFT_SHOULDER, Rig::SecondaryControllerType_LeftShoulder },
-        AnimPose pose(avatarRightArmPose.getRotation(), avatarRightArmPose.getTranslation());
+        { controller::Action::RIGHT_SHOULDER, Rig::SecondaryControllerType_RightShoulder },
-        params.controllerPoses[Rig::ControllerType_RightArm] = avatarToRigPose * pose;
+        { controller::Action::LEFT_ARM, Rig::SecondaryControllerType_LeftArm },
-        params.controllerActiveFlags[Rig::ControllerType_RightArm] = true;
+        { controller::Action::RIGHT_ARM, Rig::SecondaryControllerType_RightArm },
-    } else {
+        { controller::Action::LEFT_FORE_ARM, Rig::SecondaryControllerType_LeftForeArm },
-        params.controllerPoses[Rig::ControllerType_RightArm] = AnimPose::identity;
+        { controller::Action::RIGHT_FORE_ARM, Rig::SecondaryControllerType_RightForeArm },
-        params.controllerActiveFlags[Rig::ControllerType_RightArm] = false;
+        { controller::Action::LEFT_UP_LEG, Rig::SecondaryControllerType_LeftUpLeg },
-    }
+        { controller::Action::RIGHT_UP_LEG, Rig::SecondaryControllerType_RightUpLeg },
-    
+        { controller::Action::LEFT_LEG, Rig::SecondaryControllerType_LeftLeg },
-    auto avatarLeftArmPose = myAvatar->getLeftArmControllerPoseInAvatarFrame();
+        { controller::Action::RIGHT_LEG, Rig::SecondaryControllerType_RightLeg },
-    if (avatarLeftArmPose.isValid()) {
+        { controller::Action::LEFT_TOE_BASE, Rig::SecondaryControllerType_LeftToeBase },
-        AnimPose pose(avatarLeftArmPose.getRotation(), avatarLeftArmPose.getTranslation());
+        { controller::Action::RIGHT_TOE_BASE, Rig::SecondaryControllerType_RightToeBase }
-        params.controllerPoses[Rig::ControllerType_LeftArm] = avatarToRigPose * pose;
+    };
-        params.controllerActiveFlags[Rig::ControllerType_LeftArm] = true;
-    } else {
-        params.controllerPoses[Rig::ControllerType_LeftArm] = AnimPose::identity;
-        params.controllerActiveFlags[Rig::ControllerType_LeftArm] = false;
-    }
 
-    auto avatarLeftHandPose = myAvatar->getLeftHandControllerPoseInAvatarFrame();
+    for (auto pair : secondaryControllers) {
-    if (avatarLeftHandPose.isValid()) {
+        auto controllerPose = myAvatar->getControllerPoseInAvatarFrame(pair.first);
-        AnimPose pose(avatarLeftHandPose.getRotation(), avatarLeftHandPose.getTranslation());
+        if (controllerPose.isValid()) {
-        params.controllerPoses[Rig::ControllerType_LeftHand] = avatarToRigPose * pose;
+            AnimPose pose(controllerPose.getRotation(), controllerPose.getTranslation());
-        params.controllerActiveFlags[Rig::ControllerType_LeftHand] = true;
+            params.secondaryControllerPoses[pair.second] = avatarToRigPose * pose;
-    } else {
+            params.secondaryControllerActiveFlags[pair.second] = true;
-        params.controllerPoses[Rig::ControllerType_LeftHand] = AnimPose::identity;
+        } else {
-        params.controllerActiveFlags[Rig::ControllerType_LeftHand] = false;
+            params.secondaryControllerPoses[pair.second] = AnimPose::identity;
-    }
+            params.secondaryControllerActiveFlags[pair.second] = false;
-
+        }
-    auto avatarRightHandPose = myAvatar->getRightHandControllerPoseInAvatarFrame();
-    if (avatarRightHandPose.isValid()) {
-        AnimPose pose(avatarRightHandPose.getRotation(), avatarRightHandPose.getTranslation());
-        params.controllerPoses[Rig::ControllerType_RightHand] = avatarToRigPose * pose;
-        params.controllerActiveFlags[Rig::ControllerType_RightHand] = true;
-    } else {
-        params.controllerPoses[Rig::ControllerType_RightHand] = AnimPose::identity;
-        params.controllerActiveFlags[Rig::ControllerType_RightHand] = false;
-    }
-
-    auto avatarLeftFootPose = myAvatar->getLeftFootControllerPoseInAvatarFrame();
-    if (avatarLeftFootPose.isValid()) {
-        AnimPose pose(avatarLeftFootPose.getRotation(), avatarLeftFootPose.getTranslation());
-        params.controllerPoses[Rig::ControllerType_LeftFoot] = avatarToRigPose * pose;
-        params.controllerActiveFlags[Rig::ControllerType_LeftFoot] = true;
-    } else {
-        params.controllerPoses[Rig::ControllerType_LeftFoot] = AnimPose::identity;
-        params.controllerActiveFlags[Rig::ControllerType_LeftFoot] = false;
-    }
-
-    auto avatarRightFootPose = myAvatar->getRightFootControllerPoseInAvatarFrame();
-    if (avatarRightFootPose.isValid()) {
-        AnimPose pose(avatarRightFootPose.getRotation(), avatarRightFootPose.getTranslation());
-        params.controllerPoses[Rig::ControllerType_RightFoot] = avatarToRigPose * pose;
-        params.controllerActiveFlags[Rig::ControllerType_RightFoot] = true;
-    } else {
-        params.controllerPoses[Rig::ControllerType_RightFoot] = AnimPose::identity;
-        params.controllerActiveFlags[Rig::ControllerType_RightFoot] = false;
     }
 
     params.bodyCapsuleRadius = myAvatar->getCharacterController()->getCapsuleRadius();
@@ -175,49 +152,106 @@ void MySkeletonModel::updateRig(float deltaTime, glm::mat4 parentTransform) {
 
     _rig.updateFromEyeParameters(eyeParams);
 
-    updateFingers(myAvatar->getLeftHandFingerControllerPosesInSensorFrame());
+    updateFingers();
-    updateFingers(myAvatar->getRightHandFingerControllerPosesInSensorFrame());
 }
 
 
-void MySkeletonModel::updateFingers(const MyAvatar::FingerPosesMap& fingerPoses) {
+void MySkeletonModel::updateFingers() {
-    // Assumes that finger poses are kept in order in the poses map.
-
-    if (fingerPoses.size() == 0) {
-        return;
-}
-
-    auto posesMapItr = fingerPoses.begin();
-
-    bool isLeftHand = posesMapItr->first < (int)controller::Action::RIGHT_HAND_THUMB1;
 
     MyAvatar* myAvatar = static_cast<MyAvatar*>(_owningAvatar);
-    auto handPose = isLeftHand 
-        ? myAvatar->getLeftHandControllerPoseInSensorFrame() 
-        : myAvatar->getRightHandControllerPoseInSensorFrame();
-    auto handJointRotation = handPose.getRotation();
 
-    bool isHandValid = handPose.isValid();
+    static std::vector<std::vector<std::pair<controller::Action, QString>>> fingerChains = {
-    bool isFingerValid = false;
+        {
-    glm::quat previousJointRotation;
+            { controller::Action::LEFT_HAND, "LeftHand" },
-
+            { controller::Action::LEFT_HAND_THUMB1, "LeftHandThumb1" },
-    while (posesMapItr != fingerPoses.end()) {
+            { controller::Action::LEFT_HAND_THUMB2, "LeftHandThumb2" },
-        auto jointName = posesMapItr->second.second;
+            { controller::Action::LEFT_HAND_THUMB3, "LeftHandThumb3" },
-        if (isHandValid && jointName.right(1) == "1") {
+            { controller::Action::LEFT_HAND_THUMB4, "LeftHandThumb4" }
-            isFingerValid = posesMapItr->second.first.isValid();
+        },
-            previousJointRotation = handJointRotation;
+        {
+            { controller::Action::LEFT_HAND, "LeftHand" },
+            { controller::Action::LEFT_HAND_INDEX1, "LeftHandIndex1" },
+            { controller::Action::LEFT_HAND_INDEX2, "LeftHandIndex2" },
+            { controller::Action::LEFT_HAND_INDEX3, "LeftHandIndex3" },
+            { controller::Action::LEFT_HAND_INDEX4, "LeftHandIndex4" }
+        },
+        {
+            { controller::Action::LEFT_HAND, "LeftHand" },
+            { controller::Action::LEFT_HAND_MIDDLE1, "LeftHandMiddle1" },
+            { controller::Action::LEFT_HAND_MIDDLE2, "LeftHandMiddle2" },
+            { controller::Action::LEFT_HAND_MIDDLE3, "LeftHandMiddle3" },
+            { controller::Action::LEFT_HAND_MIDDLE4, "LeftHandMiddle4" }
+        },
+        {
+            { controller::Action::LEFT_HAND, "LeftHand" },
+            { controller::Action::LEFT_HAND_RING1, "LeftHandRing1" },
+            { controller::Action::LEFT_HAND_RING2, "LeftHandRing2" },
+            { controller::Action::LEFT_HAND_RING3, "LeftHandRing3" },
+            { controller::Action::LEFT_HAND_RING4, "LeftHandRing4" }
+        },
+        {
+            { controller::Action::LEFT_HAND, "LeftHand" },
+            { controller::Action::LEFT_HAND_PINKY1, "LeftHandPinky1" },
+            { controller::Action::LEFT_HAND_PINKY2, "LeftHandPinky2" },
+            { controller::Action::LEFT_HAND_PINKY3, "LeftHandPinky3" },
+            { controller::Action::LEFT_HAND_PINKY4, "LeftHandPinky4" }
+        },
+        {
+            { controller::Action::RIGHT_HAND, "RightHand" },
+            { controller::Action::RIGHT_HAND_THUMB1, "RightHandThumb1" },
+            { controller::Action::RIGHT_HAND_THUMB2, "RightHandThumb2" },
+            { controller::Action::RIGHT_HAND_THUMB3, "RightHandThumb3" },
+            { controller::Action::RIGHT_HAND_THUMB4, "RightHandThumb4" }
+        },
+        {
+            { controller::Action::RIGHT_HAND, "RightHand" },
+            { controller::Action::RIGHT_HAND_INDEX1, "RightHandIndex1" },
+            { controller::Action::RIGHT_HAND_INDEX2, "RightHandIndex2" },
+            { controller::Action::RIGHT_HAND_INDEX3, "RightHandIndex3" },
+            { controller::Action::RIGHT_HAND_INDEX4, "RightHandIndex4" }
+        },
+        {
+            { controller::Action::RIGHT_HAND, "RightHand" },
+            { controller::Action::RIGHT_HAND_MIDDLE1, "RightHandMiddle1" },
+            { controller::Action::RIGHT_HAND_MIDDLE2, "RightHandMiddle2" },
+            { controller::Action::RIGHT_HAND_MIDDLE3, "RightHandMiddle3" },
+            { controller::Action::RIGHT_HAND_MIDDLE4, "RightHandMiddle4" }
+        },
+        {
+            { controller::Action::RIGHT_HAND, "RightHand" },
+            { controller::Action::RIGHT_HAND_RING1, "RightHandRing1" },
+            { controller::Action::RIGHT_HAND_RING2, "RightHandRing2" },
+            { controller::Action::RIGHT_HAND_RING3, "RightHandRing3" },
+            { controller::Action::RIGHT_HAND_RING4, "RightHandRing4" }
+        },
+        {
+            { controller::Action::RIGHT_HAND, "RightHand" },
+            { controller::Action::RIGHT_HAND_PINKY1, "RightHandPinky1" },
+            { controller::Action::RIGHT_HAND_PINKY2, "RightHandPinky2" },
+            { controller::Action::RIGHT_HAND_PINKY3, "RightHandPinky3" },
+            { controller::Action::RIGHT_HAND_PINKY4, "RightHandPinky4" }
         }
+    };
 
-        if (isHandValid && isFingerValid) {
+    const float CONTROLLER_PRIORITY = 2.0f;
-            auto thisJointRotation = posesMapItr->second.first.getRotation();
+
-            const float CONTROLLER_PRIORITY = 2.0f;
+    for (auto& chain : fingerChains) {
-            _rig.setJointRotation(_rig.indexOfJoint(jointName), true, glm::inverse(previousJointRotation) * thisJointRotation, 
+        glm::quat prevAbsRot = Quaternions::IDENTITY;
-                CONTROLLER_PRIORITY);
+        for (auto& link : chain) {
-            previousJointRotation = thisJointRotation;
+            int index = _rig.indexOfJoint(link.second);
-        } else {
+            if (index >= 0) {
-            _rig.clearJointAnimationPriority(_rig.indexOfJoint(jointName));
+                auto pose = myAvatar->getControllerPoseInSensorFrame(link.first);
+                if (pose.valid) {
+                    glm::quat relRot = glm::inverse(prevAbsRot) * pose.getRotation();
+                    // only set the rotation for the finger joints, not the hands.
+                    if (link.first != controller::Action::LEFT_HAND && link.first != controller::Action::RIGHT_HAND) {
+                        _rig.setJointRotation(index, true, relRot, CONTROLLER_PRIORITY);
+                    }
+                    prevAbsRot = pose.getRotation();
+                } else {
+                    _rig.clearJointAnimationPriority(index);
+                }
+            }
         }
-
-        posesMapItr++;
     }
 }

interface/src/avatar/MySkeletonModel.h

@@ -24,7 +24,7 @@ public:
     void updateRig(float deltaTime, glm::mat4 parentTransform) override;
 
 private:
-    void updateFingers(const MyAvatar::FingerPosesMap& fingerPoses);
+    void updateFingers();
 };
 
 #endif // hifi_MySkeletonModel_h

interface/src/scripting/AudioDevices.cpp

@@ -12,6 +12,7 @@
 #include <map>
 
 #include <shared/QtHelpers.h>
+#include <plugins/DisplayPlugin.h>
 
 #include "AudioDevices.h"
 

interface/src/ui/JSConsole.cpp

@@ -9,6 +9,8 @@
 //  See the accompanying file LICENSE or http://www.apache.org/licenses/LICENSE-2.0.html
 //
 
+#include "JSConsole.h"
+
 #include <QFuture>
 #include <QKeyEvent>
 #include <QLabel>
@@ -20,7 +22,6 @@
 #include <PathUtils.h>
 
 #include "Application.h"
-#include "JSConsole.h"
 #include "ScriptHighlighting.h"
 
 const int NO_CURRENT_HISTORY_COMMAND = -1;
@@ -60,14 +61,12 @@ void _writeLines(const QString& filename, const QList<QString>& lines) {
     QTextStream(&file) << json;
 }
 
-JSConsole::JSConsole(QWidget* parent, ScriptEngine* scriptEngine) :
+JSConsole::JSConsole(QWidget* parent, const QSharedPointer<ScriptEngine>& scriptEngine) :
     QWidget(parent),
     _ui(new Ui::Console),
     _currentCommandInHistory(NO_CURRENT_HISTORY_COMMAND),
     _savedHistoryFilename(QStandardPaths::writableLocation(QStandardPaths::DataLocation) + "/" + HISTORY_FILENAME),
-    _commandHistory(_readLines(_savedHistoryFilename)),
+    _commandHistory(_readLines(_savedHistoryFilename)) {
-    _ownScriptEngine(scriptEngine == NULL),
-    _scriptEngine(NULL) {
     _ui->setupUi(this);
     _ui->promptTextEdit->setLineWrapMode(QTextEdit::NoWrap);
     _ui->promptTextEdit->setWordWrapMode(QTextOption::NoWrap);
@@ -90,36 +89,37 @@ JSConsole::JSConsole(QWidget* parent, ScriptEngine* scriptEngine) :
 }
 
 JSConsole::~JSConsole() {
-    disconnect(_scriptEngine, SIGNAL(printedMessage(const QString&)), this, SLOT(handlePrint(const QString&)));
+    if (_scriptEngine) {
-    disconnect(_scriptEngine, SIGNAL(errorMessage(const QString&)), this, SLOT(handleError(const QString&)));
+        disconnect(_scriptEngine.data(), SIGNAL(printedMessage(const QString&)), this, SLOT(handlePrint(const QString&)));
-    if (_ownScriptEngine) {
+        disconnect(_scriptEngine.data(), SIGNAL(errorMessage(const QString&)), this, SLOT(handleError(const QString&)));
-        _scriptEngine->deleteLater();
+        _scriptEngine.reset();
     }
     delete _ui;
 }
 
-void JSConsole::setScriptEngine(ScriptEngine* scriptEngine) {
+void JSConsole::setScriptEngine(const QSharedPointer<ScriptEngine>&  scriptEngine) {
     if (_scriptEngine == scriptEngine && scriptEngine != NULL) {
         return;
     }
     if (_scriptEngine != NULL) {
-        disconnect(_scriptEngine, &ScriptEngine::printedMessage, this, &JSConsole::handlePrint);
+        disconnect(_scriptEngine.data(), &ScriptEngine::printedMessage, this, &JSConsole::handlePrint);
-        disconnect(_scriptEngine, &ScriptEngine::infoMessage, this, &JSConsole::handleInfo);
+        disconnect(_scriptEngine.data(), &ScriptEngine::infoMessage, this, &JSConsole::handleInfo);
-        disconnect(_scriptEngine, &ScriptEngine::warningMessage, this, &JSConsole::handleWarning);
+        disconnect(_scriptEngine.data(), &ScriptEngine::warningMessage, this, &JSConsole::handleWarning);
-        disconnect(_scriptEngine, &ScriptEngine::errorMessage, this, &JSConsole::handleError);
+        disconnect(_scriptEngine.data(), &ScriptEngine::errorMessage, this, &JSConsole::handleError);
-        if (_ownScriptEngine) {
+        _scriptEngine.reset();
-            _scriptEngine->deleteLater();
-        }
     }
 
     // if scriptEngine is NULL then create one and keep track of it using _ownScriptEngine
-    _ownScriptEngine = (scriptEngine == NULL);
+    if (scriptEngine.isNull()) {
-    _scriptEngine = _ownScriptEngine ? DependencyManager::get<ScriptEngines>()->loadScript(_consoleFileName, false) : scriptEngine;
+        _scriptEngine = QSharedPointer<ScriptEngine>(DependencyManager::get<ScriptEngines>()->loadScript(_consoleFileName, false), &QObject::deleteLater);
+    } else {
+        _scriptEngine = scriptEngine;
+    }
 
-    connect(_scriptEngine, &ScriptEngine::printedMessage, this, &JSConsole::handlePrint);
+    connect(_scriptEngine.data(), &ScriptEngine::printedMessage, this, &JSConsole::handlePrint);
-    connect(_scriptEngine, &ScriptEngine::infoMessage, this, &JSConsole::handleInfo);
+    connect(_scriptEngine.data(), &ScriptEngine::infoMessage, this, &JSConsole::handleInfo);
-    connect(_scriptEngine, &ScriptEngine::warningMessage, this, &JSConsole::handleWarning);
+    connect(_scriptEngine.data(), &ScriptEngine::warningMessage, this, &JSConsole::handleWarning);
-    connect(_scriptEngine, &ScriptEngine::errorMessage, this, &JSConsole::handleError);
+    connect(_scriptEngine.data(), &ScriptEngine::errorMessage, this, &JSConsole::handleError);
 }
 
 void JSConsole::executeCommand(const QString& command) {
@@ -135,19 +135,22 @@ void JSConsole::executeCommand(const QString& command) {
 
     appendMessage(">", "<span style='" + COMMAND_STYLE + "'>" + command.toHtmlEscaped() + "</span>");
 
-    QFuture<QScriptValue> future = QtConcurrent::run(this, &JSConsole::executeCommandInWatcher, command);
+    QWeakPointer<ScriptEngine> weakScriptEngine = _scriptEngine;
+    auto consoleFileName = _consoleFileName;
+    QFuture<QScriptValue> future = QtConcurrent::run([weakScriptEngine, consoleFileName, command]()->QScriptValue{
+        QScriptValue result;
+        auto scriptEngine = weakScriptEngine.lock();
+        if (scriptEngine) {
+            BLOCKING_INVOKE_METHOD(scriptEngine.data(), "evaluate",
+                Q_RETURN_ARG(QScriptValue, result),
+                Q_ARG(const QString&, command),
+                Q_ARG(const QString&, consoleFileName));
+        }
+        return result;
+    });
     _executeWatcher.setFuture(future);
 }
 
-QScriptValue JSConsole::executeCommandInWatcher(const QString& command) {
-    QScriptValue result;
-    BLOCKING_INVOKE_METHOD(_scriptEngine, "evaluate",
-                              Q_RETURN_ARG(QScriptValue, result),
-                              Q_ARG(const QString&, command),
-                              Q_ARG(const QString&, _consoleFileName));
-    return result;
-}
-
 void JSConsole::commandFinished() {
     QScriptValue result = _executeWatcher.result();
 

interface/src/ui/JSConsole.h

@@ -17,6 +17,7 @@
 #include <QFutureWatcher>
 #include <QObject>
 #include <QWidget>
+#include <QSharedPointer>
 
 #include "ui_console.h"
 #include "ScriptEngine.h"
@@ -29,10 +30,10 @@ const int CONSOLE_HEIGHT = 200;
 class JSConsole : public QWidget {
     Q_OBJECT
 public:
-    JSConsole(QWidget* parent, ScriptEngine* scriptEngine = NULL);
+    JSConsole(QWidget* parent, const QSharedPointer<ScriptEngine>& scriptEngine = QSharedPointer<ScriptEngine>());
     ~JSConsole();
 
-    void setScriptEngine(ScriptEngine* scriptEngine = NULL);
+    void setScriptEngine(const QSharedPointer<ScriptEngine>& scriptEngine = QSharedPointer<ScriptEngine>());
     void clear();
 
 public slots:
@@ -58,17 +59,14 @@ private:
     void setToNextCommandInHistory();
     void setToPreviousCommandInHistory();
     void resetCurrentCommandHistory();
-    QScriptValue executeCommandInWatcher(const QString& command);
 
     QFutureWatcher<QScriptValue> _executeWatcher;
     Ui::Console* _ui;
     int _currentCommandInHistory;
     QString _savedHistoryFilename;
     QList<QString> _commandHistory;
-    // Keeps track if the script engine is created inside the JSConsole
-    bool _ownScriptEngine;
     QString _rootCommand;
-    ScriptEngine* _scriptEngine;
+    QSharedPointer<ScriptEngine> _scriptEngine;
     static const QString _consoleFileName;
 };
 

interface/src/ui/PreferencesDialog.cpp

@@ -106,6 +106,12 @@ void setupPreferences() {
         auto setter = [](bool value) { qApp->setPreferAvatarFingerOverStylus(value); };
         preferences->addPreference(new CheckPreference(UI_CATEGORY, "Prefer Avatar Finger Over Stylus", getter, setter));
     }
+    {
+        static const QString RETICLE_ICON_NAME = { Cursor::Manager::getIconName(Cursor::Icon::RETICLE) };
+        auto getter = []()->bool { return qApp->getPreferredCursor() == RETICLE_ICON_NAME; };
+        auto setter = [](bool value) { qApp->setPreferredCursor(value ? RETICLE_ICON_NAME : QString()); };
+        preferences->addPreference(new CheckPreference(UI_CATEGORY, "Use reticle cursor instead of arrow", getter, setter));
+    }
 
     // Snapshots
     static const QString SNAPSHOTS { "Snapshots" };
@@ -181,6 +187,11 @@ void setupPreferences() {
         preference->setStep(1);
         preferences->addPreference(preference);
     }
+    {
+        auto getter = [=]()->QString { return myAvatar->getDominantHand(); };
+        auto setter = [=](const QString& value) { myAvatar->setDominantHand(value); };
+        preferences->addPreference(new PrimaryHandPreference(AVATAR_TUNING, "Dominant Hand", getter, setter));
+    }
     {
         auto getter = [=]()->float { return myAvatar->getUniformScale(); };
         auto setter = [=](float value) { myAvatar->setTargetScale(value); };
@@ -300,17 +311,6 @@ void setupPreferences() {
         preferences->addPreference(preference);
     }
 
-
-    {
-        auto getter = []()->float { return controller::InputDevice::getReticleMoveSpeed(); };
-        auto setter = [](float value) { controller::InputDevice::setReticleMoveSpeed(value); };
-        auto preference = new SpinnerPreference("Sixense Controllers", "Reticle movement speed", getter, setter);
-        preference->setMin(0);
-        preference->setMax(100);
-        preference->setStep(1);
-        preferences->addPreference(preference);
-    }
-
     {
         static const QString RENDER("Graphics");
         auto renderConfig = qApp->getRenderEngine()->getConfiguration();

interface/src/ui/ResourceImageItem.cpp

@@ -8,7 +8,6 @@
 // See the accompanying file LICENSE or http://www.apache.org/licenses/LICENSE-2.0.html
 //
 
-//#include "Application.h"
 #include "ResourceImageItem.h"
 
 #include <QOpenGLFramebufferObjectFormat>
@@ -16,6 +15,8 @@
 #include <QOpenGLExtraFunctions>
 #include <QOpenGLContext>
 
+#include <plugins/DisplayPlugin.h>
+
 ResourceImageItem::ResourceImageItem() : QQuickFramebufferObject() {
     auto textureCache = DependencyManager::get<TextureCache>();
     connect(textureCache.data(), SIGNAL(spectatorCameraFramebufferReset()), this, SLOT(update()));

interface/src/ui/SnapshotAnimated.cpp

@@ -13,8 +13,9 @@
 #include <QtCore/QObject>
 #include <QtCore/QString>
 #include <QtGui/QImage>
+#include <QtConcurrent/QtConcurrentRun>
 
-#include <QtConcurrent/qtconcurrentrun.h>
+#include <plugins/DisplayPlugin.h>
 #include "SnapshotAnimated.h"
 
 QTimer* SnapshotAnimated::snapshotAnimatedTimer = NULL;

interface/src/ui/TestingDialog.cpp

@@ -24,12 +24,12 @@ TestingDialog::TestingDialog(QWidget* parent) :
     _console->setFixedHeight(TESTING_CONSOLE_HEIGHT);
 
     auto _engines = DependencyManager::get<ScriptEngines>();
-    _engine = _engines->loadScript(qApp->applicationDirPath() + testRunnerRelativePath);
+    _engine.reset(_engines->loadScript(qApp->applicationDirPath() + testRunnerRelativePath));
     _console->setScriptEngine(_engine);
-    connect(_engine, &ScriptEngine::finished, this, &TestingDialog::onTestingFinished);
+    connect(_engine.data(), &ScriptEngine::finished, this, &TestingDialog::onTestingFinished);
 }
 
 void TestingDialog::onTestingFinished(const QString& scriptPath) {
-    _engine = nullptr;
+    _engine.reset();
-    _console->setScriptEngine(nullptr);
+    _console->setScriptEngine();
 }

interface/src/ui/TestingDialog.h

@@ -29,7 +29,7 @@ public:
 
 private:
     std::unique_ptr<JSConsole> _console;
-    ScriptEngine* _engine;
+    QSharedPointer<ScriptEngine> _engine;
 };
 
 #endif

interface/src/ui/overlays/ContextOverlayInterface.cpp

@@ -0,0 +1,279 @@
+//
+//  ContextOverlayInterface.cpp
+//  interface/src/ui/overlays
+//
+//  Created by Zach Fox on 2017-07-14.
+//  Copyright 2017 High Fidelity, Inc.
+//
+//  Distributed under the Apache License, Version 2.0.
+//  See the accompanying file LICENSE or http://www.apache.org/licenses/LICENSE-2.0.html
+//
+
+#include "ContextOverlayInterface.h"
+#include "Application.h"
+
+#include <EntityTreeRenderer.h>
+
+static const float CONTEXT_OVERLAY_TABLET_OFFSET = 30.0f; // Degrees
+static const float CONTEXT_OVERLAY_TABLET_ORIENTATION = 210.0f; // Degrees
+static const float CONTEXT_OVERLAY_TABLET_DISTANCE = 0.65F; // Meters
+ContextOverlayInterface::ContextOverlayInterface() {
+    // "context_overlay" debug log category disabled by default.
+    // Create your own "qtlogging.ini" file and set your "QT_LOGGING_CONF" environment variable
+    // if you'd like to enable/disable certain categories.
+    // More details: http://doc.qt.io/qt-5/qloggingcategory.html#configuring-categories
+    QLoggingCategory::setFilterRules(QStringLiteral("hifi.context_overlay.debug=false"));
+
+    _entityScriptingInterface = DependencyManager::get<EntityScriptingInterface>();
+    _hmdScriptingInterface = DependencyManager::get<HMDScriptingInterface>();
+    _tabletScriptingInterface = DependencyManager::get<TabletScriptingInterface>();
+
+    _entityPropertyFlags += PROP_POSITION;
+    _entityPropertyFlags += PROP_ROTATION;
+    _entityPropertyFlags += PROP_MARKETPLACE_ID;
+    _entityPropertyFlags += PROP_DIMENSIONS;
+    _entityPropertyFlags += PROP_REGISTRATION_POINT;
+
+    // initially, set _enabled to match the switch.  Later we enable/disable via the getter/setters
+    // if we are in edit or pal (for instance).  Note this is temporary, as we expect to enable this all
+    // the time after getting edge highlighting, etc...
+    _enabled = _settingSwitch.get();
+
+    auto entityTreeRenderer = DependencyManager::get<EntityTreeRenderer>().data();
+    connect(entityTreeRenderer, SIGNAL(mousePressOnEntity(const EntityItemID&, const PointerEvent&)), this, SLOT(createOrDestroyContextOverlay(const EntityItemID&, const PointerEvent&)));
+    connect(entityTreeRenderer, SIGNAL(hoverEnterEntity(const EntityItemID&, const PointerEvent&)), this, SLOT(contextOverlays_hoverEnterEntity(const EntityItemID&, const PointerEvent&)));
+    connect(entityTreeRenderer, SIGNAL(hoverLeaveEntity(const EntityItemID&, const PointerEvent&)), this, SLOT(contextOverlays_hoverLeaveEntity(const EntityItemID&, const PointerEvent&)));
+    connect(_tabletScriptingInterface->getTablet("com.highfidelity.interface.tablet.system"), &TabletProxy::tabletShownChanged, this, [&]() {
+        if (_contextOverlayJustClicked && _hmdScriptingInterface->isMounted()) {
+            QUuid tabletFrameID = _hmdScriptingInterface->getCurrentTabletFrameID();
+            auto myAvatar = DependencyManager::get<AvatarManager>()->getMyAvatar();
+            glm::quat cameraOrientation = qApp->getCamera().getOrientation();
+            QVariantMap props;
+            props.insert("position", vec3toVariant(myAvatar->getEyePosition() + glm::quat(glm::radians(glm::vec3(0.0f, CONTEXT_OVERLAY_TABLET_OFFSET, 0.0f))) * (CONTEXT_OVERLAY_TABLET_DISTANCE * (cameraOrientation * Vectors::FRONT))));
+            props.insert("orientation", quatToVariant(cameraOrientation * glm::quat(glm::radians(glm::vec3(0.0f, CONTEXT_OVERLAY_TABLET_ORIENTATION, 0.0f)))));
+            qApp->getOverlays().editOverlay(tabletFrameID, props);
+            _contextOverlayJustClicked = false;
+        }
+    });
+}
+
+static const uint32_t LEFT_HAND_HW_ID = 1;
+static const xColor CONTEXT_OVERLAY_COLOR = { 255, 255, 255 };
+static const float CONTEXT_OVERLAY_INSIDE_DISTANCE = 1.0f; // in meters
+static const float CONTEXT_OVERLAY_CLOSE_DISTANCE = 1.5f; // in meters
+static const float CONTEXT_OVERLAY_CLOSE_SIZE = 0.12f; // in meters, same x and y dims
+static const float CONTEXT_OVERLAY_FAR_SIZE = 0.08f; // in meters, same x and y dims
+static const float CONTEXT_OVERLAY_CLOSE_OFFSET_ANGLE = 20.0f;
+static const float CONTEXT_OVERLAY_UNHOVERED_ALPHA = 0.85f;
+static const float CONTEXT_OVERLAY_HOVERED_ALPHA = 1.0f;
+static const float CONTEXT_OVERLAY_UNHOVERED_PULSEMIN = 0.6f;
+static const float CONTEXT_OVERLAY_UNHOVERED_PULSEMAX = 1.0f;
+static const float CONTEXT_OVERLAY_UNHOVERED_PULSEPERIOD = 1.0f;
+static const float CONTEXT_OVERLAY_UNHOVERED_COLORPULSE = 1.0f;
+static const float CONTEXT_OVERLAY_FAR_OFFSET = 0.1f;
+
+void ContextOverlayInterface::setEnabled(bool enabled) {
+    // only enable/disable if the setting in 'on'.   If it is 'off',
+    // make sure _enabled is always false.
+    if (_settingSwitch.get()) {
+        _enabled = enabled;
+    } else {
+        _enabled = false;
+    }
+}
+
+bool ContextOverlayInterface::createOrDestroyContextOverlay(const EntityItemID& entityItemID, const PointerEvent& event) {
+    if (_enabled && event.getButton() == PointerEvent::SecondaryButton) {
+        if (contextOverlayFilterPassed(entityItemID)) {
+            qCDebug(context_overlay) << "Creating Context Overlay on top of entity with ID: " << entityItemID;
+
+            // Add all necessary variables to the stack
+            EntityItemProperties entityProperties = _entityScriptingInterface->getEntityProperties(entityItemID, _entityPropertyFlags);
+            glm::vec3 cameraPosition = qApp->getCamera().getPosition();
+            float distanceFromCameraToEntity = glm::distance(entityProperties.getPosition(), cameraPosition);
+            glm::vec3 entityDimensions = entityProperties.getDimensions();
+            glm::vec3 entityPosition = entityProperties.getPosition();
+            glm::vec3 contextOverlayPosition = entityProperties.getPosition();
+            glm::vec2 contextOverlayDimensions;
+
+            // Update the position of the overlay if the registration point of the entity
+            // isn't default
+            if (entityProperties.getRegistrationPoint() != glm::vec3(0.5f)) {
+                glm::vec3 adjustPos = entityProperties.getRegistrationPoint() - glm::vec3(0.5f);
+                entityPosition = entityPosition - (entityProperties.getRotation() * (adjustPos * entityProperties.getDimensions()));
+            }
+
+            enableEntityHighlight(entityItemID);
+
+            AABox boundingBox = AABox(entityPosition - (entityDimensions / 2.0f), entityDimensions * 2.0f);
+
+            // Update the cached Entity Marketplace ID
+            _entityMarketplaceID = entityProperties.getMarketplaceID();
+
+
+            if (!_currentEntityWithContextOverlay.isNull() && _currentEntityWithContextOverlay != entityItemID) {
+                disableEntityHighlight(_currentEntityWithContextOverlay);
+            }
+
+            // Update the cached "Current Entity with Context Overlay" variable
+            setCurrentEntityWithContextOverlay(entityItemID);
+
+            // Here, we determine the position and dimensions of the Context Overlay.
+            if (boundingBox.contains(cameraPosition)) {
+                // If the camera is inside the box...
+                // ...position the Context Overlay 1 meter in front of the camera.
+                contextOverlayPosition = cameraPosition + CONTEXT_OVERLAY_INSIDE_DISTANCE * (qApp->getCamera().getOrientation() * Vectors::FRONT);
+                contextOverlayDimensions = glm::vec2(CONTEXT_OVERLAY_CLOSE_SIZE, CONTEXT_OVERLAY_CLOSE_SIZE) * glm::distance(contextOverlayPosition, cameraPosition);
+            } else if (distanceFromCameraToEntity < CONTEXT_OVERLAY_CLOSE_DISTANCE) {
+                // Else if the entity is too close to the camera...
+                // ...rotate the Context Overlay to the right of the entity.
+                // This makes it easy to inspect things you're holding.
+                float offsetAngle = -CONTEXT_OVERLAY_CLOSE_OFFSET_ANGLE;
+                if (event.getID() == LEFT_HAND_HW_ID) {
+                    offsetAngle *= -1;
+                }
+                contextOverlayPosition = (glm::quat(glm::radians(glm::vec3(0.0f, offsetAngle, 0.0f))) * (entityPosition - cameraPosition)) + cameraPosition;
+                contextOverlayDimensions = glm::vec2(CONTEXT_OVERLAY_CLOSE_SIZE, CONTEXT_OVERLAY_CLOSE_SIZE) * glm::distance(contextOverlayPosition, cameraPosition);
+            } else {
+                // Else, place the Context Overlay some offset away from the entity's bounding
+                // box in the direction of the camera.
+                glm::vec3 direction = glm::normalize(entityPosition - cameraPosition);
+                float distance;
+                BoxFace face;
+                glm::vec3 normal;
+                boundingBox.findRayIntersection(cameraPosition, direction, distance, face, normal);
+                contextOverlayPosition = (cameraPosition + direction * distance) - direction * CONTEXT_OVERLAY_FAR_OFFSET;
+                contextOverlayDimensions = glm::vec2(CONTEXT_OVERLAY_FAR_SIZE, CONTEXT_OVERLAY_FAR_SIZE) * glm::distance(contextOverlayPosition, cameraPosition);
+            }
+
+            // Finally, setup and draw the Context Overlay
+            if (_contextOverlayID == UNKNOWN_OVERLAY_ID || !qApp->getOverlays().isAddedOverlay(_contextOverlayID)) {
+                _contextOverlay = std::make_shared<Image3DOverlay>();
+                _contextOverlay->setAlpha(CONTEXT_OVERLAY_UNHOVERED_ALPHA);
+                _contextOverlay->setPulseMin(CONTEXT_OVERLAY_UNHOVERED_PULSEMIN);
+                _contextOverlay->setPulseMax(CONTEXT_OVERLAY_UNHOVERED_PULSEMAX);
+                _contextOverlay->setColorPulse(CONTEXT_OVERLAY_UNHOVERED_COLORPULSE);
+                _contextOverlay->setIgnoreRayIntersection(false);
+                _contextOverlay->setDrawInFront(true);
+                _contextOverlay->setURL(PathUtils::resourcesPath() + "images/inspect-icon.png");
+                _contextOverlay->setIsFacingAvatar(true);
+                _contextOverlayID = qApp->getOverlays().addOverlay(_contextOverlay);
+            }
+            _contextOverlay->setPosition(contextOverlayPosition);
+            _contextOverlay->setDimensions(contextOverlayDimensions);
+            _contextOverlay->setRotation(entityProperties.getRotation());
+            _contextOverlay->setVisible(true);
+
+            return true;
+        }
+    } else {
+        if (!_currentEntityWithContextOverlay.isNull()) {
+            return destroyContextOverlay(_currentEntityWithContextOverlay, event);
+        }
+        return false;
+    }
+    return false;
+}
+
+bool ContextOverlayInterface::contextOverlayFilterPassed(const EntityItemID& entityItemID) {
+    EntityItemProperties entityProperties = _entityScriptingInterface->getEntityProperties(entityItemID, _entityPropertyFlags);
+    return (entityProperties.getMarketplaceID().length() != 0);
+}
+
+bool ContextOverlayInterface::destroyContextOverlay(const EntityItemID& entityItemID, const PointerEvent& event) {
+    if (_contextOverlayID != UNKNOWN_OVERLAY_ID) {
+        qCDebug(context_overlay) << "Destroying Context Overlay on top of entity with ID: " << entityItemID;
+        disableEntityHighlight(entityItemID);
+        setCurrentEntityWithContextOverlay(QUuid());
+        _entityMarketplaceID.clear();
+        // Destroy the Context Overlay
+        qApp->getOverlays().deleteOverlay(_contextOverlayID);
+        _contextOverlay = NULL;
+        _contextOverlayID = UNKNOWN_OVERLAY_ID;
+        return true;
+    }
+    return false;
+}
+
+bool ContextOverlayInterface::destroyContextOverlay(const EntityItemID& entityItemID) {
+    return ContextOverlayInterface::destroyContextOverlay(entityItemID, PointerEvent());
+}
+
+void ContextOverlayInterface::contextOverlays_mousePressOnOverlay(const OverlayID& overlayID, const PointerEvent& event) {
+    if (overlayID == _contextOverlayID  && event.getButton() == PointerEvent::PrimaryButton) {
+        qCDebug(context_overlay) << "Clicked Context Overlay. Entity ID:" << _currentEntityWithContextOverlay << "Overlay ID:" << overlayID;
+        openMarketplace();
+        destroyContextOverlay(_currentEntityWithContextOverlay, PointerEvent());
+        _contextOverlayJustClicked = true;
+    }
+}
+
+void ContextOverlayInterface::contextOverlays_hoverEnterOverlay(const OverlayID& overlayID, const PointerEvent& event) {
+    if (_contextOverlayID != UNKNOWN_OVERLAY_ID && _contextOverlay) {
+        qCDebug(context_overlay) << "Started hovering over Context Overlay. Overlay ID:" << overlayID;
+        _contextOverlay->setColor(CONTEXT_OVERLAY_COLOR);
+        _contextOverlay->setColorPulse(0.0f); // pulse off
+        _contextOverlay->setPulsePeriod(0.0f); // pulse off
+        _contextOverlay->setAlpha(CONTEXT_OVERLAY_HOVERED_ALPHA);
+    }
+}
+
+void ContextOverlayInterface::contextOverlays_hoverLeaveOverlay(const OverlayID& overlayID, const PointerEvent& event) {
+    if (_contextOverlayID != UNKNOWN_OVERLAY_ID && _contextOverlay) {
+        qCDebug(context_overlay) << "Stopped hovering over Context Overlay. Overlay ID:" << overlayID;
+        _contextOverlay->setColor(CONTEXT_OVERLAY_COLOR);
+        _contextOverlay->setColorPulse(CONTEXT_OVERLAY_UNHOVERED_COLORPULSE);
+        _contextOverlay->setPulsePeriod(CONTEXT_OVERLAY_UNHOVERED_PULSEPERIOD);
+        _contextOverlay->setAlpha(CONTEXT_OVERLAY_UNHOVERED_ALPHA);
+    }
+}
+
+void ContextOverlayInterface::contextOverlays_hoverEnterEntity(const EntityItemID& entityID, const PointerEvent& event) {
+    if (contextOverlayFilterPassed(entityID)) {
+        enableEntityHighlight(entityID);
+    }
+}
+
+void ContextOverlayInterface::contextOverlays_hoverLeaveEntity(const EntityItemID& entityID, const PointerEvent& event) {
+    if (_currentEntityWithContextOverlay != entityID) {
+        disableEntityHighlight(entityID);
+    }
+}
+
+static const QString MARKETPLACE_BASE_URL = "https://metaverse.highfidelity.com/marketplace/items/";
+
+void ContextOverlayInterface::openMarketplace() {
+    // lets open the tablet and go to the current item in
+    // the marketplace (if the current entity has a
+    // marketplaceID)
+    if (!_currentEntityWithContextOverlay.isNull() && _entityMarketplaceID.length() > 0) {
+        auto tablet = dynamic_cast<TabletProxy*>(_tabletScriptingInterface->getTablet("com.highfidelity.interface.tablet.system"));
+        // construct the url to the marketplace item
+        QString url = MARKETPLACE_BASE_URL + _entityMarketplaceID;
+        tablet->gotoWebScreen(url);
+        _hmdScriptingInterface->openTablet();
+        _isInMarketplaceInspectionMode = true;
+    }
+}
+
+void ContextOverlayInterface::enableEntityHighlight(const EntityItemID& entityItemID) {
+    auto entityTree = qApp->getEntities()->getTree();
+    entityTree->withReadLock([&] {
+        auto entityItem = entityTree->findEntityByEntityItemID(entityItemID);
+        if ((entityItem != NULL) && !entityItem->getShouldHighlight()) {
+            qCDebug(context_overlay) << "Setting 'shouldHighlight' to 'true' for Entity ID:" << entityItemID;
+            entityItem->setShouldHighlight(true);
+        }
+    });
+}
+
+void ContextOverlayInterface::disableEntityHighlight(const EntityItemID& entityItemID) {
+    auto entityTree = qApp->getEntities()->getTree();
+    entityTree->withReadLock([&] {
+        auto entityItem = entityTree->findEntityByEntityItemID(entityItemID);
+        if ((entityItem != NULL) && entityItem->getShouldHighlight()) {
+            qCDebug(context_overlay) << "Setting 'shouldHighlight' to 'false' for Entity ID:" << entityItemID;
+            entityItem->setShouldHighlight(false);
+        }
+    });
+}

interface/src/ui/overlays/ContextOverlayInterface.h

@@ -0,0 +1,85 @@
+//
+//  ContextOverlayInterface.h
+//  interface/src/ui/overlays
+//
+//  Created by Zach Fox on 2017-07-14.
+//  Copyright 2017 High Fidelity, Inc.
+//
+//  Distributed under the Apache License, Version 2.0.
+//  See the accompanying file LICENSE or http://www.apache.org/licenses/LICENSE-2.0.html
+//
+
+#pragma once
+#ifndef hifi_ContextOverlayInterface_h
+#define hifi_ContextOverlayInterface_h
+
+#include <QtCore/QObject>
+#include <QUuid>
+
+#include <DependencyManager.h>
+#include <PointerEvent.h>
+#include <ui/TabletScriptingInterface.h>
+#include "avatar/AvatarManager.h"
+
+#include "EntityScriptingInterface.h"
+#include "ui/overlays/Image3DOverlay.h"
+#include "ui/overlays/Overlays.h"
+#include "scripting/HMDScriptingInterface.h"
+
+#include "EntityTree.h"
+#include "ContextOverlayLogging.h"
+
+/**jsdoc
+* @namespace ContextOverlay
+*/
+class ContextOverlayInterface : public QObject, public Dependency  {
+    Q_OBJECT
+
+    Q_PROPERTY(QUuid entityWithContextOverlay READ getCurrentEntityWithContextOverlay WRITE setCurrentEntityWithContextOverlay)
+    Q_PROPERTY(bool enabled READ getEnabled WRITE setEnabled)
+    Q_PROPERTY(bool isInMarketplaceInspectionMode READ getIsInMarketplaceInspectionMode WRITE setIsInMarketplaceInspectionMode)
+    QSharedPointer<EntityScriptingInterface> _entityScriptingInterface;
+    EntityPropertyFlags _entityPropertyFlags;
+    QSharedPointer<HMDScriptingInterface> _hmdScriptingInterface;
+    QSharedPointer<TabletScriptingInterface> _tabletScriptingInterface;
+    OverlayID _contextOverlayID { UNKNOWN_OVERLAY_ID };
+    std::shared_ptr<Image3DOverlay> _contextOverlay { nullptr };
+public:
+    ContextOverlayInterface();
+
+    Q_INVOKABLE QUuid getCurrentEntityWithContextOverlay() { return _currentEntityWithContextOverlay; }
+    void setCurrentEntityWithContextOverlay(const QUuid& entityID) { _currentEntityWithContextOverlay = entityID; }
+    void setEnabled(bool enabled);
+    bool getEnabled() { return _enabled; }
+    bool getIsInMarketplaceInspectionMode() { return _isInMarketplaceInspectionMode; }
+    void setIsInMarketplaceInspectionMode(bool mode) { _isInMarketplaceInspectionMode = mode; }
+
+public slots:
+    bool createOrDestroyContextOverlay(const EntityItemID& entityItemID, const PointerEvent& event);
+    bool destroyContextOverlay(const EntityItemID& entityItemID, const PointerEvent& event);
+    bool destroyContextOverlay(const EntityItemID& entityItemID);
+    void contextOverlays_mousePressOnOverlay(const OverlayID& overlayID, const PointerEvent& event);
+    void contextOverlays_hoverEnterOverlay(const OverlayID& overlayID, const PointerEvent& event);
+    void contextOverlays_hoverLeaveOverlay(const OverlayID& overlayID, const PointerEvent& event);
+    void contextOverlays_hoverEnterEntity(const EntityItemID& entityID, const PointerEvent& event);
+    void contextOverlays_hoverLeaveEntity(const EntityItemID& entityID, const PointerEvent& event);
+    bool contextOverlayFilterPassed(const EntityItemID& entityItemID);
+
+private:
+    bool _verboseLogging { true };
+    bool _enabled { true };
+    QUuid _currentEntityWithContextOverlay{};
+    QString _entityMarketplaceID;
+    bool _contextOverlayJustClicked { false };
+
+    bool _isInMarketplaceInspectionMode { false };
+
+    Setting::Handle<bool> _settingSwitch { "inspectionMode", false };
+
+    void openMarketplace();
+    void enableEntityHighlight(const EntityItemID& entityItemID);
+    void disableEntityHighlight(const EntityItemID& entityItemID);
+
+};
+
+#endif // hifi_ContextOverlayInterface_h

interface/src/ui/overlays/ContextOverlayLogging.cpp

@@ -1,6 +1,6 @@
 //
-//  HoverOverlayLogging.cpp
+//  ContextOverlayLogging.cpp
-//  libraries/entities/src
+//  interface/src/ui/overlays
 //
 //  Created by Zach Fox on 2017-07-17
 //  Copyright 2017 High Fidelity, Inc.
@@ -9,6 +9,6 @@
 //  See the accompanying file LICENSE or http://www.apache.org/licenses/LICENSE-2.0.html
 //
 
-#include "HoverOverlayLogging.h"
+#include "ContextOverlayLogging.h"
 
-Q_LOGGING_CATEGORY(hover_overlay, "hifi.hover_overlay")
+Q_LOGGING_CATEGORY(context_overlay, "hifi.context_overlay")

interface/src/ui/overlays/ContextOverlayLogging.h

@@ -1,6 +1,6 @@
 //
-//  HoverOverlayLogging.h
+//  ContextOverlayLogging.h
-//  libraries/entities/src
+//  interface/src/ui/overlays
 //
 //  Created by Zach Fox on 2017-07-17
 //  Copyright 2017 High Fidelity, Inc.
@@ -10,11 +10,11 @@
 //
 
 #pragma once
-#ifndef hifi_HoverOverlayLogging_h
+#ifndef hifi_ContextOverlayLogging_h
-#define hifi_HoverOverlayLogging_h
+#define hifi_ContextOverlayLogging_h
 
 #include <QLoggingCategory>
 
-Q_DECLARE_LOGGING_CATEGORY(hover_overlay)
+Q_DECLARE_LOGGING_CATEGORY(context_overlay)
 
-#endif // hifi_HoverOverlayLogging_h
+#endif // hifi_ContextOverlayLogging_h

interface/src/ui/overlays/Overlay.cpp

@@ -20,7 +20,7 @@ Overlay::Overlay() :
     _renderItemID(render::Item::INVALID_ITEM_ID),
     _isLoaded(true),
     _alpha(DEFAULT_ALPHA),
-    _pulse(0.0f),
+    _pulse(1.0f),
     _pulseMax(0.0f),
     _pulseMin(0.0f),
     _pulsePeriod(1.0f),

interface/src/ui/overlays/Overlays.cpp

@@ -116,7 +116,7 @@ void Overlays::renderHUD(RenderArgs* renderArgs) {
 
     QMutexLocker locker(&_mutex);
     foreach(Overlay::Pointer thisOverlay, _overlaysHUD) {
-    
+
         // Reset all batch pipeline settings between overlay
         geometryCache->useSimpleDrawPipeline(batch);
         batch.setResourceTexture(0, textureCache->getWhiteTexture()); // FIXME - do we really need to do this??
@@ -136,7 +136,7 @@ void Overlays::enable() {
     _enabled = true;
 }
 
-// Note, can't be invoked by scripts, but can be called by the InterfaceParentFinder 
+// Note, can't be invoked by scripts, but can be called by the InterfaceParentFinder
 // class on packet processing threads
 Overlay::Pointer Overlays::getOverlay(OverlayID id) const {
     QMutexLocker locker(&_mutex);
@@ -244,8 +244,8 @@ OverlayID Overlays::cloneOverlay(OverlayID id) {
 
 bool Overlays::editOverlay(OverlayID id, const QVariant& properties) {
     if (QThread::currentThread() != thread()) {
-        // NOTE editOverlay can be called very frequently in scripts and can't afford to 
+        // NOTE editOverlay can be called very frequently in scripts and can't afford to
-        // block waiting on the main thread.  Additionally, no script actually 
+        // block waiting on the main thread.  Additionally, no script actually
         // examines the return value and does something useful with it, so use a non-blocking
         // invoke and just always return true
         QMetaObject::invokeMethod(this, "editOverlay", Q_ARG(OverlayID, id), Q_ARG(QVariant, properties));
@@ -705,28 +705,28 @@ bool Overlays::isAddedOverlay(OverlayID id) {
     return _overlaysHUD.contains(id) || _overlaysWorld.contains(id);
 }
 
-void Overlays::sendMousePressOnOverlay(OverlayID overlayID, const PointerEvent& event) {
+void Overlays::sendMousePressOnOverlay(const OverlayID& overlayID, const PointerEvent& event) {
-    emit mousePressOnOverlay(overlayID, event);
+    QMetaObject::invokeMethod(this, "mousePressOnOverlay", Q_ARG(OverlayID, overlayID), Q_ARG(PointerEvent, event));
 }
 
-void Overlays::sendMouseReleaseOnOverlay(OverlayID overlayID, const PointerEvent& event) {
+void Overlays::sendMouseReleaseOnOverlay(const OverlayID& overlayID, const PointerEvent& event) {
-    emit mouseReleaseOnOverlay(overlayID, event);
+    QMetaObject::invokeMethod(this, "mouseReleaseOnOverlay", Q_ARG(OverlayID, overlayID), Q_ARG(PointerEvent, event));
 }
 
-void Overlays::sendMouseMoveOnOverlay(OverlayID overlayID, const PointerEvent& event) {
+void Overlays::sendMouseMoveOnOverlay(const OverlayID& overlayID, const PointerEvent& event) {
-    emit mouseMoveOnOverlay(overlayID, event);
+    QMetaObject::invokeMethod(this, "mouseMoveOnOverlay", Q_ARG(OverlayID, overlayID), Q_ARG(PointerEvent, event));
 }
 
-void Overlays::sendHoverEnterOverlay(OverlayID id, PointerEvent event) {
+void Overlays::sendHoverEnterOverlay(const OverlayID& id, const PointerEvent& event) {
-    emit hoverEnterOverlay(id, event);
+    QMetaObject::invokeMethod(this, "hoverEnterOverlay", Q_ARG(OverlayID, id), Q_ARG(PointerEvent, event));
 }
 
-void Overlays::sendHoverOverOverlay(OverlayID  id, PointerEvent event) {
+void Overlays::sendHoverOverOverlay(const OverlayID& id, const PointerEvent& event) {
-    emit hoverOverOverlay(id, event);
+    QMetaObject::invokeMethod(this, "hoverOverOverlay", Q_ARG(OverlayID, id), Q_ARG(PointerEvent, event));
 }
 
-void Overlays::sendHoverLeaveOverlay(OverlayID  id, PointerEvent event) {
+void Overlays::sendHoverLeaveOverlay(const OverlayID& id, const PointerEvent& event) {
-    emit hoverLeaveOverlay(id, event);
+    QMetaObject::invokeMethod(this, "hoverLeaveOverlay", Q_ARG(OverlayID, id), Q_ARG(PointerEvent, event));
 }
 
 OverlayID Overlays::getKeyboardFocusOverlay() {
@@ -775,7 +775,7 @@ float Overlays::height() {
 
 static const uint32_t MOUSE_POINTER_ID = 0;
 
-static glm::vec2 projectOntoOverlayXYPlane(glm::vec3 position, glm::quat rotation, glm::vec2 dimensions, const PickRay& pickRay, 
+static glm::vec2 projectOntoOverlayXYPlane(glm::vec3 position, glm::quat rotation, glm::vec2 dimensions, const PickRay& pickRay,
     const RayToOverlayIntersectionResult& rayPickResult) {
 
     // Project the intersection point onto the local xy plane of the overlay.
@@ -818,15 +818,20 @@ static PointerEvent::Button toPointerButton(const QMouseEvent& event) {
     }
 }
 
-PointerEvent Overlays::calculatePointerEvent(Overlay::Pointer overlay, PickRay ray,
+PointerEvent Overlays::calculateOverlayPointerEvent(OverlayID overlayID, PickRay ray,
                                              RayToOverlayIntersectionResult rayPickResult, QMouseEvent* event,
                                              PointerEvent::EventType eventType) {
+    auto overlay = std::dynamic_pointer_cast<Planar3DOverlay>(getOverlay(overlayID));
+    if (getOverlayType(overlayID) == "web3d") {
+        overlay = std::dynamic_pointer_cast<Web3DOverlay>(getOverlay(overlayID));
+    }
+    if (!overlay) {
+        return PointerEvent();
+    }
+    glm::vec3 position = overlay->getPosition();
+    glm::quat rotation = overlay->getRotation();
+    glm::vec2 dimensions = overlay->getSize();
 
-    auto thisOverlay = std::dynamic_pointer_cast<Web3DOverlay>(overlay);
-
-    auto position = thisOverlay->getPosition();
-    auto rotation = thisOverlay->getRotation();
-    auto dimensions = thisOverlay->getSize();
 
     glm::vec2 pos2D = projectOntoOverlayXYPlane(position, rotation, dimensions, ray, rayPickResult);
 
@@ -874,13 +879,9 @@ bool Overlays::mousePressEvent(QMouseEvent* event) {
     if (rayPickResult.intersects) {
         _currentClickingOnOverlayID = rayPickResult.overlayID;
 
-        // Only Web overlays can have focus.
+        PointerEvent pointerEvent = calculateOverlayPointerEvent(_currentClickingOnOverlayID, ray, rayPickResult, event, PointerEvent::Press);
-        auto thisOverlay = std::dynamic_pointer_cast<Web3DOverlay>(getOverlay(_currentClickingOnOverlayID));
+        emit mousePressOnOverlay(_currentClickingOnOverlayID, pointerEvent);
-        if (thisOverlay) {
+        return true;
-            auto pointerEvent = calculatePointerEvent(thisOverlay, ray, rayPickResult, event, PointerEvent::Press);
-            emit mousePressOnOverlay(_currentClickingOnOverlayID, pointerEvent);
-            return true;
-        }
     }
     emit mousePressOffOverlay();
     return false;
@@ -894,13 +895,9 @@ bool Overlays::mouseDoublePressEvent(QMouseEvent* event) {
     if (rayPickResult.intersects) {
         _currentClickingOnOverlayID = rayPickResult.overlayID;
 
-        // Only Web overlays can have focus.
+        auto pointerEvent = calculateOverlayPointerEvent(_currentClickingOnOverlayID, ray, rayPickResult, event, PointerEvent::Press);
-        auto thisOverlay = std::dynamic_pointer_cast<Web3DOverlay>(getOverlay(_currentClickingOnOverlayID));
+        emit mouseDoublePressOnOverlay(_currentClickingOnOverlayID, pointerEvent);
-        if (thisOverlay) {
+        return true;
-            auto pointerEvent = calculatePointerEvent(thisOverlay, ray, rayPickResult, event, PointerEvent::Press);
-            emit mouseDoublePressOnOverlay(_currentClickingOnOverlayID, pointerEvent);
-            return true;
-        }
     }
     emit mouseDoublePressOffOverlay();
     return false;
@@ -912,13 +909,8 @@ bool Overlays::mouseReleaseEvent(QMouseEvent* event) {
     PickRay ray = qApp->computePickRay(event->x(), event->y());
     RayToOverlayIntersectionResult rayPickResult = findRayIntersectionForMouseEvent(ray);
     if (rayPickResult.intersects) {
-
+        auto pointerEvent = calculateOverlayPointerEvent(rayPickResult.overlayID, ray, rayPickResult, event, PointerEvent::Release);
-        // Only Web overlays can have focus.
+        emit mouseReleaseOnOverlay(rayPickResult.overlayID, pointerEvent);
-        auto thisOverlay = std::dynamic_pointer_cast<Web3DOverlay>(getOverlay(rayPickResult.overlayID));
-        if (thisOverlay) {
-            auto pointerEvent = calculatePointerEvent(thisOverlay, ray, rayPickResult, event, PointerEvent::Release);
-            emit mouseReleaseOnOverlay(rayPickResult.overlayID, pointerEvent);
-        }
     }
 
     _currentClickingOnOverlayID = UNKNOWN_OVERLAY_ID;
@@ -931,40 +923,29 @@ bool Overlays::mouseMoveEvent(QMouseEvent* event) {
     PickRay ray = qApp->computePickRay(event->x(), event->y());
     RayToOverlayIntersectionResult rayPickResult = findRayIntersectionForMouseEvent(ray);
     if (rayPickResult.intersects) {
+        auto pointerEvent = calculateOverlayPointerEvent(rayPickResult.overlayID, ray, rayPickResult, event, PointerEvent::Move);
+        emit mouseMoveOnOverlay(rayPickResult.overlayID, pointerEvent);
 
-        // Only Web overlays can have focus.
+        // If previously hovering over a different overlay then leave hover on that overlay.
-        auto thisOverlay = std::dynamic_pointer_cast<Web3DOverlay>(getOverlay(rayPickResult.overlayID));
+        if (_currentHoverOverOverlayID != UNKNOWN_OVERLAY_ID && rayPickResult.overlayID != _currentHoverOverOverlayID) {
-        if (thisOverlay) {
+            auto pointerEvent = calculateOverlayPointerEvent(_currentHoverOverOverlayID, ray, rayPickResult, event, PointerEvent::Move);
-            auto pointerEvent = calculatePointerEvent(thisOverlay, ray, rayPickResult, event, PointerEvent::Move);
+            emit hoverLeaveOverlay(_currentHoverOverOverlayID, pointerEvent);
-            emit mouseMoveOnOverlay(rayPickResult.overlayID, pointerEvent);
-
-            // If previously hovering over a different overlay then leave hover on that overlay.
-            if (_currentHoverOverOverlayID != UNKNOWN_OVERLAY_ID && rayPickResult.overlayID != _currentHoverOverOverlayID) {
-                auto thisOverlay = std::dynamic_pointer_cast<Web3DOverlay>(getOverlay(_currentHoverOverOverlayID));
-                if (thisOverlay) {
-                    auto pointerEvent = calculatePointerEvent(thisOverlay, ray, rayPickResult, event, PointerEvent::Move);
-                    emit hoverLeaveOverlay(_currentHoverOverOverlayID, pointerEvent);
-                }
-            }
-
-            // If hovering over a new overlay then enter hover on that overlay.
-            if (rayPickResult.overlayID != _currentHoverOverOverlayID) {
-                emit hoverEnterOverlay(rayPickResult.overlayID, pointerEvent);
-            }
-
-            // Hover over current overlay.
-            emit hoverOverOverlay(rayPickResult.overlayID, pointerEvent);
-
-            _currentHoverOverOverlayID = rayPickResult.overlayID;
         }
+
+        // If hovering over a new overlay then enter hover on that overlay.
+        if (rayPickResult.overlayID != _currentHoverOverOverlayID) {
+            emit hoverEnterOverlay(rayPickResult.overlayID, pointerEvent);
+        }
+
+        // Hover over current overlay.
+        emit hoverOverOverlay(rayPickResult.overlayID, pointerEvent);
+
+        _currentHoverOverOverlayID = rayPickResult.overlayID;
     } else {
         // If previously hovering an overlay then leave hover.
         if (_currentHoverOverOverlayID != UNKNOWN_OVERLAY_ID) {
-            auto thisOverlay = std::dynamic_pointer_cast<Web3DOverlay>(getOverlay(_currentHoverOverOverlayID));
+            auto pointerEvent = calculateOverlayPointerEvent(_currentHoverOverOverlayID, ray, rayPickResult, event, PointerEvent::Move);
-            if (thisOverlay) {
+            emit hoverLeaveOverlay(_currentHoverOverOverlayID, pointerEvent);
-                auto pointerEvent = calculatePointerEvent(thisOverlay, ray, rayPickResult, event, PointerEvent::Move);
-                emit hoverLeaveOverlay(_currentHoverOverOverlayID, pointerEvent);
-            }
 
             _currentHoverOverOverlayID = UNKNOWN_OVERLAY_ID;
         }

interface/src/ui/overlays/Overlays.h

@@ -131,7 +131,7 @@ public slots:
     OverlayID cloneOverlay(OverlayID id);
 
     /**jsdoc
-     * Edit an overlay's properties. 
+     * Edit an overlay's properties.
      *
      * @function Overlays.editOverlay
      * @param {Overlays.OverlayID} overlayID The ID of the overlay to edit.
@@ -288,13 +288,13 @@ public slots:
 
 #endif
 
-    void sendMousePressOnOverlay(OverlayID overlayID, const PointerEvent& event);
+    void sendMousePressOnOverlay(const OverlayID& overlayID, const PointerEvent& event);
-    void sendMouseReleaseOnOverlay(OverlayID overlayID, const PointerEvent& event);
+    void sendMouseReleaseOnOverlay(const OverlayID& overlayID, const PointerEvent& event);
-    void sendMouseMoveOnOverlay(OverlayID overlayID, const PointerEvent& event);
+    void sendMouseMoveOnOverlay(const OverlayID& overlayID, const PointerEvent& event);
 
-    void sendHoverEnterOverlay(OverlayID id, PointerEvent event);
+    void sendHoverEnterOverlay(const OverlayID& id, const PointerEvent& event);
-    void sendHoverOverOverlay(OverlayID id, PointerEvent event);
+    void sendHoverOverOverlay(const OverlayID& id, const PointerEvent& event);
-    void sendHoverLeaveOverlay(OverlayID id, PointerEvent event);
+    void sendHoverLeaveOverlay(const OverlayID& id, const PointerEvent& event);
 
     OverlayID getKeyboardFocusOverlay();
     void setKeyboardFocusOverlay(OverlayID id);
@@ -323,7 +323,7 @@ signals:
 private:
     void cleanupOverlaysToDelete();
 
-    mutable QMutex _mutex;
+    mutable QMutex _mutex { QMutex::Recursive };
     QMap<OverlayID, Overlay::Pointer> _overlaysHUD;
     QMap<OverlayID, Overlay::Pointer> _overlaysWorld;
 #if OVERLAY_PANELS
@@ -337,7 +337,7 @@ private:
 #endif
     bool _enabled = true;
 
-    PointerEvent calculatePointerEvent(Overlay::Pointer overlay, PickRay ray, RayToOverlayIntersectionResult rayPickResult,
+    PointerEvent calculateOverlayPointerEvent(OverlayID overlayID, PickRay ray, RayToOverlayIntersectionResult rayPickResult,
         QMouseEvent* event, PointerEvent::EventType eventType);
 
     OverlayID _currentClickingOnOverlayID { UNKNOWN_OVERLAY_ID };

interface/src/ui/overlays/Planar3DOverlay.h

@@ -21,6 +21,7 @@ public:
     Planar3DOverlay(const Planar3DOverlay* planar3DOverlay);
     
     virtual AABox getBounds() const override;
+    virtual glm::vec2 getSize() const { return _dimensions; };
     
     glm::vec2 getDimensions() const { return _dimensions; }
     void setDimensions(float value) { _dimensions = glm::vec2(value); }

interface/src/ui/overlays/Web3DOverlay.cpp

@@ -100,30 +100,14 @@ Web3DOverlay::~Web3DOverlay() {
         }
 
         _webSurface->pause();
-        _webSurface->disconnect(_connection);
+        auto overlays = &(qApp->getOverlays());
-
+        QObject::disconnect(overlays, &Overlays::mousePressOnOverlay, this, nullptr);
-        QObject::disconnect(_mousePressConnection);
+        QObject::disconnect(overlays, &Overlays::mouseReleaseOnOverlay, this, nullptr);
-        _mousePressConnection = QMetaObject::Connection();
+        QObject::disconnect(overlays, &Overlays::mouseMoveOnOverlay, this, nullptr);
-        QObject::disconnect(_mouseReleaseConnection);
+        QObject::disconnect(overlays, &Overlays::hoverLeaveOverlay, this, nullptr);
-        _mouseReleaseConnection = QMetaObject::Connection();
+        QObject::disconnect(this, &Web3DOverlay::scriptEventReceived, _webSurface.data(), &OffscreenQmlSurface::emitScriptEvent);
-        QObject::disconnect(_mouseMoveConnection);
+        QObject::disconnect(_webSurface.data(), &OffscreenQmlSurface::webEventReceived, this, &Web3DOverlay::webEventReceived);
-        _mouseMoveConnection = QMetaObject::Connection();
+        DependencyManager::get<OffscreenQmlSurfaceCache>()->release(QML, _webSurface);
-        QObject::disconnect(_hoverLeaveConnection);
-        _hoverLeaveConnection = QMetaObject::Connection();
-
-        QObject::disconnect(_emitScriptEventConnection);
-        _emitScriptEventConnection = QMetaObject::Connection();
-        QObject::disconnect(_webEventReceivedConnection);
-        _webEventReceivedConnection = QMetaObject::Connection();
-
-        // The lifetime of the QML surface MUST be managed by the main thread
-        // Additionally, we MUST use local variables copied by value, rather than
-        // member variables, since they would implicitly refer to a this that
-        // is no longer valid
-        auto webSurface = _webSurface;
-        AbstractViewStateInterface::instance()->postLambdaEvent([webSurface] {
-            DependencyManager::get<OffscreenQmlSurfaceCache>()->release(QML, webSurface);
-        });
         _webSurface.reset();
     }
     auto geometryCache = DependencyManager::get<GeometryCache>();
@@ -143,8 +127,9 @@ QString Web3DOverlay::pickURL() {
     QUrl sourceUrl(_url);
     if (sourceUrl.scheme() == "http" || sourceUrl.scheme() == "https" ||
         _url.toLower().endsWith(".htm") || _url.toLower().endsWith(".html")) {
-
+        if (_webSurface) {
-        _webSurface->setBaseUrl(QUrl::fromLocalFile(PathUtils::resourcesPath() + "/qml/"));
+            _webSurface->setBaseUrl(QUrl::fromLocalFile(PathUtils::resourcesPath() + "/qml/"));
+        }
         return "Web3DOverlay.qml";
     } else {
         return QUrl::fromLocalFile(PathUtils::resourcesPath()).toString() + "/" + _url;
@@ -153,6 +138,9 @@ QString Web3DOverlay::pickURL() {
 
 
 void Web3DOverlay::loadSourceURL() {
+    if (!_webSurface) {
+        return;
+    }
 
     QUrl sourceUrl(_url);
     if (sourceUrl.scheme() == "http" || sourceUrl.scheme() == "https" ||
@@ -205,6 +193,7 @@ void Web3DOverlay::loadSourceURL() {
             _webSurface->getSurfaceContext()->setContextProperty("SoundCache", DependencyManager::get<SoundCache>().data());
 
             _webSurface->getSurfaceContext()->setContextProperty("pathToFonts", "../../");
+
             tabletScriptingInterface->setQmlTabletRoot("com.highfidelity.interface.tablet.system", _webSurface.data());
 
             // mark the TabletProxy object as cpp ownership.
@@ -252,23 +241,24 @@ void Web3DOverlay::render(RenderArgs* args) {
             }
         };
 
-        _mousePressConnection = connect(&(qApp->getOverlays()), &Overlays::mousePressOnOverlay, this, forwardPointerEvent, Qt::DirectConnection);
+        auto overlays = &(qApp->getOverlays());
-        _mouseReleaseConnection = connect(&(qApp->getOverlays()), &Overlays::mouseReleaseOnOverlay, this, forwardPointerEvent, Qt::DirectConnection);
+        QObject::connect(overlays, &Overlays::mousePressOnOverlay, this, forwardPointerEvent);
-        _mouseMoveConnection = connect(&(qApp->getOverlays()), &Overlays::mouseMoveOnOverlay, this, forwardPointerEvent, Qt::DirectConnection);
+        QObject::connect(overlays, &Overlays::mouseReleaseOnOverlay, this, forwardPointerEvent);
-        _hoverLeaveConnection = connect(&(qApp->getOverlays()), &Overlays::hoverLeaveOverlay, this, [=](OverlayID overlayID, const PointerEvent& event) {
+        QObject::connect(overlays, &Overlays::mouseMoveOnOverlay, this, forwardPointerEvent);
+        QObject::connect(overlays, &Overlays::hoverLeaveOverlay, this, [=](OverlayID overlayID, const PointerEvent& event) {
             auto self = weakSelf.lock();
             if (!self) {
                 return;
             }
-            if (self->_pressed && overlayID == selfOverlayID) {
+            if (overlayID == selfOverlayID && (self->_pressed || (!self->_activeTouchPoints.empty() && self->_touchBeginAccepted))) {
                 PointerEvent endEvent(PointerEvent::Release, event.getID(), event.getPos2D(), event.getPos3D(), event.getNormal(), event.getDirection(),
                                       event.getButton(), event.getButtons(), event.getKeyboardModifiers());
-                forwardPointerEvent(overlayID, event);
+                forwardPointerEvent(overlayID, endEvent);
             }
-        }, Qt::DirectConnection);
+        });
 
-        _emitScriptEventConnection = connect(this, &Web3DOverlay::scriptEventReceived, _webSurface.data(), &OffscreenQmlSurface::emitScriptEvent);
+        QObject::connect(this, &Web3DOverlay::scriptEventReceived, _webSurface.data(), &OffscreenQmlSurface::emitScriptEvent);
-        _webEventReceivedConnection = connect(_webSurface.data(), &OffscreenQmlSurface::webEventReceived, this, &Web3DOverlay::webEventReceived);
+        QObject::connect(_webSurface.data(), &OffscreenQmlSurface::webEventReceived, this, &Web3DOverlay::webEventReceived);
     } else {
         if (_currentMaxFPS != _desiredMaxFPS) {
             setMaxFPS(_desiredMaxFPS);
@@ -361,19 +351,68 @@ void Web3DOverlay::handlePointerEventAsTouch(const PointerEvent& event) {
         return;
     }
 
-    glm::vec2 windowPos = event.getPos2D() * (METERS_TO_INCHES * _dpi);
+    //do not send secondary button events to tablet
-    QPointF windowPoint(windowPos.x, windowPos.y);
+    if (event.getButton() == PointerEvent::SecondaryButton ||
-
+        //do not block composed events
-    if (event.getType() == PointerEvent::Press && event.getButton() == PointerEvent::PrimaryButton) {
+        event.getButtons() == PointerEvent::SecondaryButton) {
-        this->_pressed = true;
+        return;
-    } else if (event.getType() == PointerEvent::Release && event.getButton() == PointerEvent::PrimaryButton) {
-        this->_pressed = false;
     }
 
-    QEvent::Type touchType;
-    Qt::TouchPointState touchPointState;
-    QEvent::Type mouseType;
 
+    QPointF windowPoint;
+    {
+        glm::vec2 windowPos = event.getPos2D() * (METERS_TO_INCHES * _dpi);
+        windowPoint = QPointF(windowPos.x, windowPos.y);
+    }
+
+    Qt::TouchPointState state = Qt::TouchPointStationary;
+    if (event.getType() == PointerEvent::Press && event.getButton() == PointerEvent::PrimaryButton) {
+        state = Qt::TouchPointPressed;
+    } else if (event.getType() == PointerEvent::Release) {
+        state = Qt::TouchPointReleased;
+    } else if (_activeTouchPoints.count(event.getID()) && windowPoint != _activeTouchPoints[event.getID()].pos()) {
+        state = Qt::TouchPointMoved;
+    }
+
+    QEvent::Type touchType = QEvent::TouchUpdate;
+    if (_activeTouchPoints.empty()) {
+        // If the first active touch point is being created, send a begin
+        touchType = QEvent::TouchBegin;
+    } if (state == Qt::TouchPointReleased && _activeTouchPoints.size() == 1 && _activeTouchPoints.count(event.getID())) {
+        // If the last active touch point is being released, send an end
+        touchType = QEvent::TouchEnd;
+    } 
+
+    {
+        QTouchEvent::TouchPoint point;
+        point.setId(event.getID());
+        point.setState(state);
+        point.setPos(windowPoint);
+        point.setScreenPos(windowPoint);
+        _activeTouchPoints[event.getID()] = point;
+    }
+
+    QTouchEvent touchEvent(touchType, &_touchDevice, event.getKeyboardModifiers());
+    {
+        QList<QTouchEvent::TouchPoint> touchPoints;
+        Qt::TouchPointStates touchPointStates;
+        for (const auto& entry : _activeTouchPoints) {
+            touchPointStates |= entry.second.state();
+            touchPoints.push_back(entry.second);
+        }
+
+        touchEvent.setWindow(_webSurface->getWindow());
+        touchEvent.setTarget(_webSurface->getRootItem());
+        touchEvent.setTouchPoints(touchPoints);
+        touchEvent.setTouchPointStates(touchPointStates);
+    }
+
+    // Send mouse events to the Web surface so that HTML dialog elements work with mouse press and hover.
+    // FIXME: Scroll bar dragging is a bit unstable in the tablet (content can jump up and down at times).
+    // This may be improved in Qt 5.8. Release notes: "Cleaned up touch and mouse event delivery".
+    //
+    // In Qt 5.9 mouse events must be sent before touch events to make sure some QtQuick components will
+    // receive mouse events
     Qt::MouseButton button = Qt::NoButton;
     Qt::MouseButtons buttons = Qt::NoButton;
     if (event.getButton() == PointerEvent::PrimaryButton) {
@@ -383,85 +422,28 @@ void Web3DOverlay::handlePointerEventAsTouch(const PointerEvent& event) {
         buttons |= Qt::LeftButton;
     }
 
-    switch (event.getType()) {
-        case PointerEvent::Press:
-            touchType = QEvent::TouchBegin;
-            touchPointState = Qt::TouchPointPressed;
-            mouseType = QEvent::MouseButtonPress;
-            break;
-        case PointerEvent::Release:
-            touchType = QEvent::TouchEnd;
-            touchPointState = Qt::TouchPointReleased;
-            mouseType = QEvent::MouseButtonRelease;
-            break;
-        case PointerEvent::Move:
-            touchType = QEvent::TouchUpdate;
-            touchPointState = Qt::TouchPointMoved;
-            mouseType = QEvent::MouseMove;
-
-            if (((event.getButtons() & PointerEvent::PrimaryButton) > 0) != this->_pressed) {
-                // Mouse was pressed/released while off the overlay; convert touch and mouse events to press/release to reflect
-                // current mouse/touch status.
-                this->_pressed = !this->_pressed;
-                if (this->_pressed) {
-                    touchType = QEvent::TouchBegin;
-                    touchPointState = Qt::TouchPointPressed;
-                    mouseType = QEvent::MouseButtonPress;
-
-                } else {
-                    touchType = QEvent::TouchEnd;
-                    touchPointState = Qt::TouchPointReleased;
-                    mouseType = QEvent::MouseButtonRelease;
-
-                }
-                button = Qt::LeftButton;
-                buttons |= Qt::LeftButton;
-            }
-
-            break;
-        default:
-            return;
-    }
-
-    //do not send secondary button events to tablet
-    if (event.getButton() == PointerEvent::SecondaryButton ||
-            //do not block composed events
-            event.getButtons() == PointerEvent::SecondaryButton) {
-        return;
-    }
-
-    QTouchEvent::TouchPoint point;
-    point.setId(event.getID());
-    point.setState(touchPointState);
-    point.setPos(windowPoint);
-    point.setScreenPos(windowPoint);
-    QList<QTouchEvent::TouchPoint> touchPoints;
-    touchPoints.push_back(point);
-
-    QTouchEvent* touchEvent = new QTouchEvent(touchType, &_touchDevice, event.getKeyboardModifiers());
-    touchEvent->setWindow(_webSurface->getWindow());
-    touchEvent->setTarget(_webSurface->getRootItem());
-    touchEvent->setTouchPoints(touchPoints);
-    touchEvent->setTouchPointStates(touchPointState);
-
-    // Send mouse events to the Web surface so that HTML dialog elements work with mouse press and hover.
-    // FIXME: Scroll bar dragging is a bit unstable in the tablet (content can jump up and down at times).
-    // This may be improved in Qt 5.8. Release notes: "Cleaned up touch and mouse event delivery".
-    //
-    // In Qt 5.9 mouse events must be sent before touch events to make sure some QtQuick components will
-    // receive mouse events
 #if QT_VERSION >= QT_VERSION_CHECK(5, 9, 0)
     if (event.getType() == PointerEvent::Move) {
-        QMouseEvent* mouseEvent = new QMouseEvent(mouseType, windowPoint, windowPoint, windowPoint, button, buttons, Qt::NoModifier);
+        QMouseEvent mouseEvent(QEvent::MouseMove, windowPoint, windowPoint, windowPoint, button, buttons, Qt::NoModifier);
-        QCoreApplication::postEvent(_webSurface->getWindow(), mouseEvent);
+        QCoreApplication::sendEvent(_webSurface->getWindow(), &mouseEvent);
     }
 #endif
-    QCoreApplication::postEvent(_webSurface->getWindow(), touchEvent);
+
+    if (touchType == QEvent::TouchBegin) {
+        _touchBeginAccepted = QCoreApplication::sendEvent(_webSurface->getWindow(), &touchEvent);
+    } else if (_touchBeginAccepted) {
+        QCoreApplication::sendEvent(_webSurface->getWindow(), &touchEvent);
+    }
+
+    // If this was a release event, remove the point from the active touch points
+    if (state == Qt::TouchPointReleased) {
+        _activeTouchPoints.erase(event.getID());
+    }
 
 #if QT_VERSION < QT_VERSION_CHECK(5, 9, 0)
     if (event.getType() == PointerEvent::Move) {
-        QMouseEvent* mouseEvent = new QMouseEvent(mouseType, windowPoint, windowPoint, windowPoint, button, buttons, Qt::NoModifier);
+        QMouseEvent mouseEvent(QEvent::MouseMove, windowPoint, windowPoint, windowPoint, button, buttons, Qt::NoModifier);
-        QCoreApplication::postEvent(_webSurface->getWindow(), mouseEvent);
+        QCoreApplication::sendEvent(_webSurface->getWindow(), &mouseEvent);
     }
 #endif
 }
@@ -505,8 +487,8 @@ void Web3DOverlay::handlePointerEventAsMouse(const PointerEvent& event) {
             return;
     }
 
-    QMouseEvent* mouseEvent = new QMouseEvent(type, windowPoint, windowPoint, windowPoint, button, buttons, Qt::NoModifier);
+    QMouseEvent mouseEvent(type, windowPoint, windowPoint, windowPoint, button, buttons, Qt::NoModifier);
-    QCoreApplication::postEvent(_webSurface->getWindow(), mouseEvent);
+    QCoreApplication::sendEvent(_webSurface->getWindow(), &mouseEvent);
 }
 
 void Web3DOverlay::setProperties(const QVariantMap& properties) {
@@ -608,12 +590,15 @@ void Web3DOverlay::setScriptURL(const QString& scriptURL) {
     _scriptURL = scriptURL;
     if (_webSurface) {
         AbstractViewStateInterface::instance()->postLambdaEvent([this, scriptURL] {
+            if (!_webSurface) {
+                return;
+            }
             _webSurface->getRootItem()->setProperty("scriptURL", scriptURL);
         });
     }
 }
 
-glm::vec2 Web3DOverlay::getSize() {
+glm::vec2 Web3DOverlay::getSize() const {
     return _resolution / _dpi * INCHES_TO_METERS * getDimensions();
 };
 
@@ -631,9 +616,5 @@ Web3DOverlay* Web3DOverlay::createClone() const {
 }
 
 void Web3DOverlay::emitScriptEvent(const QVariant& message) {
-    if (QThread::currentThread() != thread()) {
+    QMetaObject::invokeMethod(this, "scriptEventReceived", Q_ARG(QVariant, message));
-        QMetaObject::invokeMethod(this, "emitScriptEvent", Qt::QueuedConnection, Q_ARG(QVariant, message));
-    } else {
-        emit scriptEventReceived(message);
-    }
 }

interface/src/ui/overlays/Web3DOverlay.h

@@ -50,7 +50,7 @@ public:
     void setProperties(const QVariantMap& properties) override;
     QVariant getProperty(const QString& property) override;
 
-    glm::vec2 getSize();
+    glm::vec2 getSize() const override;
 
     virtual bool findRayIntersection(const glm::vec3& origin, const glm::vec3& direction, float& distance,
         BoxFace& face, glm::vec3& surfaceNormal) override;
@@ -72,7 +72,6 @@ signals:
 private:
     InputMode _inputMode { Touch };
     QSharedPointer<OffscreenQmlSurface> _webSurface;
-    QMetaObject::Connection _connection;
     gpu::TexturePointer _texture;
     QString _url;
     QString _scriptURL;
@@ -82,20 +81,14 @@ private:
     bool _showKeyboardFocusHighlight{ true };
 
     bool _pressed{ false };
+    bool _touchBeginAccepted { false };
+    std::map<uint32_t, QTouchEvent::TouchPoint> _activeTouchPoints;
     QTouchDevice _touchDevice;
 
     uint8_t _desiredMaxFPS { 10 };
     uint8_t _currentMaxFPS { 0 };
 
     bool _mayNeedResize { false };
-
-    QMetaObject::Connection _mousePressConnection;
-    QMetaObject::Connection _mouseReleaseConnection;
-    QMetaObject::Connection _mouseMoveConnection;
-    QMetaObject::Connection _hoverLeaveConnection;
-
-    QMetaObject::Connection _emitScriptEventConnection;
-    QMetaObject::Connection _webEventReceivedConnection;
 };
 
 #endif // hifi_Web3DOverlay_h

libraries/animation/src/AnimInverseKinematics.cpp

@@ -23,12 +23,13 @@
 #include "CubicHermiteSpline.h"
 #include "AnimUtil.h"
 
+static const int MAX_TARGET_MARKERS = 30;
 static const float JOINT_CHAIN_INTERP_TIME = 0.25f;
 
 static void lookupJointInfo(const AnimInverseKinematics::JointChainInfo& jointChainInfo,
-                                 int indexA, int indexB,
+                            int indexA, int indexB,
-                                 const AnimInverseKinematics::JointInfo** jointInfoA,
+                            const AnimInverseKinematics::JointInfo** jointInfoA,
-                                 const AnimInverseKinematics::JointInfo** jointInfoB) {
+                            const AnimInverseKinematics::JointInfo** jointInfoB) {
     *jointInfoA = nullptr;
     *jointInfoB = nullptr;
     for (size_t i = 0; i < jointChainInfo.jointInfoVec.size(); i++) {
@@ -97,6 +98,12 @@ AnimInverseKinematics::~AnimInverseKinematics() {
     _rotationAccumulators.clear();
     _translationAccumulators.clear();
     _targetVarVec.clear();
+
+    // remove markers
+    for (int i = 0; i < MAX_TARGET_MARKERS; i++) {
+        QString name = QString("ikTarget%1").arg(i);
+        DebugDraw::getInstance().removeMyAvatarMarker(name);
+    }
 }
 
 void AnimInverseKinematics::loadDefaultPoses(const AnimPoseVec& poses) {
@@ -1015,19 +1022,30 @@ const AnimPoseVec& AnimInverseKinematics::overlay(const AnimVariantMap& animVars
                 // debug render ik targets
                 if (context.getEnableDebugDrawIKTargets()) {
                     const vec4 WHITE(1.0f);
+                    const vec4 GREEN(0.0f, 1.0f, 0.0f, 1.0f);
                     glm::mat4 rigToAvatarMat = createMatFromQuatAndPos(Quaternions::Y_180, glm::vec3());
+                    int targetNum = 0;
 
                     for (auto& target : targets) {
                         glm::mat4 geomTargetMat = createMatFromQuatAndPos(target.getRotation(), target.getTranslation());
                         glm::mat4 avatarTargetMat = rigToAvatarMat * context.getGeometryToRigMatrix() * geomTargetMat;
 
-                        QString name = QString("ikTarget%1").arg(target.getIndex());
+                        QString name = QString("ikTarget%1").arg(targetNum);
                         DebugDraw::getInstance().addMyAvatarMarker(name, glmExtractRotation(avatarTargetMat), extractTranslation(avatarTargetMat), WHITE);
+                        targetNum++;
+                    }
+
+                    // draw secondary ik targets
+                    for (auto& iter : _secondaryTargetsInRigFrame) {
+                        glm::mat4 avatarTargetMat = rigToAvatarMat * (glm::mat4)iter.second;
+                        QString name = QString("ikTarget%1").arg(targetNum);
+                        DebugDraw::getInstance().addMyAvatarMarker(name, glmExtractRotation(avatarTargetMat), extractTranslation(avatarTargetMat), GREEN);
+                        targetNum++;
                     }
                 } else if (context.getEnableDebugDrawIKTargets() != _previousEnableDebugIKTargets) {
                     // remove markers if they were added last frame.
-                    for (auto& target : targets) {
+                    for (int i = 0; i < MAX_TARGET_MARKERS; i++) {
-                        QString name = QString("ikTarget%1").arg(target.getIndex());
+                        QString name = QString("ikTarget%1").arg(i);
                         DebugDraw::getInstance().removeMyAvatarMarker(name);
                     }
                 }
@@ -1038,7 +1056,9 @@ const AnimPoseVec& AnimInverseKinematics::overlay(const AnimVariantMap& animVars
             {
                 PROFILE_RANGE_EX(simulation_animation, "ik/ccd", 0xffff00ff, 0);
 
+                setSecondaryTargets(context);
                 preconditionRelativePosesToAvoidLimbLock(context, targets);
+
                 solve(context, targets, dt, jointChainInfoVec);
             }
 
@@ -1125,6 +1145,22 @@ void AnimInverseKinematics::clearIKJointLimitHistory() {
     }
 }
 
+void AnimInverseKinematics::setSecondaryTargetInRigFrame(int jointIndex, const AnimPose& pose) {
+    auto iter = _secondaryTargetsInRigFrame.find(jointIndex);
+    if (iter != _secondaryTargetsInRigFrame.end()) {
+        iter->second = pose;
+    } else {
+        _secondaryTargetsInRigFrame.insert({ jointIndex, pose });
+    }
+}
+
+void AnimInverseKinematics::clearSecondaryTarget(int jointIndex) {
+    auto iter = _secondaryTargetsInRigFrame.find(jointIndex);
+    if (iter != _secondaryTargetsInRigFrame.end()) {
+        _secondaryTargetsInRigFrame.erase(iter);
+    }
+}
+
 RotationConstraint* AnimInverseKinematics::getConstraint(int index) const {
     RotationConstraint* constraint = nullptr;
     std::map<int, RotationConstraint*>::const_iterator constraintItr = _constraints.find(index);
@@ -1575,7 +1611,7 @@ void AnimInverseKinematics::debugDrawRelativePoses(const AnimContext& context) c
     const vec4 GREEN(0.0f, 1.0f, 0.0f, 1.0f);
     const vec4 BLUE(0.0f, 0.0f, 1.0f, 1.0f);
     const vec4 GRAY(0.2f, 0.2f, 0.2f, 1.0f);
-    const float AXIS_LENGTH = 2.0f; // cm
+    const float AXIS_LENGTH = 10.0f; // cm
 
     // draw each pose
     for (int i = 0; i < (int)poses.size(); i++) {
@@ -1605,8 +1641,10 @@ void AnimInverseKinematics::debugDrawIKChain(const JointChainInfo& jointChainInf
     // copy debug joint rotations into the relative poses
     for (size_t i = 0; i < jointChainInfo.jointInfoVec.size(); i++) {
         const JointInfo& info = jointChainInfo.jointInfoVec[i];
-        poses[info.jointIndex].rot() = info.rot;
+        if (info.jointIndex != _hipsIndex) {
-        poses[info.jointIndex].trans() = info.trans;
+            poses[info.jointIndex].rot() = info.rot;
+            poses[info.jointIndex].trans() = info.trans;
+        }
     }
 
     // convert relative poses to absolute
@@ -1825,6 +1863,59 @@ void AnimInverseKinematics::preconditionRelativePosesToAvoidLimbLock(const AnimC
     }
 }
 
+// overwrites _relativePoses with secondary poses.
+void AnimInverseKinematics::setSecondaryTargets(const AnimContext& context) {
+
+    if (_secondaryTargetsInRigFrame.empty()) {
+        return;
+    }
+
+    // special case for arm secondary poses.
+    // determine if shoulder joint should look-at position of arm joint.
+    bool shoulderShouldLookAtArm = false;
+    const int leftArmIndex = _skeleton->nameToJointIndex("LeftArm");
+    const int rightArmIndex = _skeleton->nameToJointIndex("RightArm");
+    const int leftShoulderIndex = _skeleton->nameToJointIndex("LeftShoulder");
+    const int rightShoulderIndex = _skeleton->nameToJointIndex("RightShoulder");
+    for (auto& iter : _secondaryTargetsInRigFrame) {
+        if (iter.first == leftShoulderIndex || iter.first == rightShoulderIndex) {
+            shoulderShouldLookAtArm = true;
+            break;
+        }
+    }
+
+    AnimPose rigToGeometryPose = AnimPose(glm::inverse(context.getGeometryToRigMatrix()));
+    for (auto& iter : _secondaryTargetsInRigFrame) {
+        AnimPose absPose = rigToGeometryPose * iter.second;
+        absPose.scale() = glm::vec3(1.0f);
+
+        AnimPose parentAbsPose;
+        int parentIndex = _skeleton->getParentIndex(iter.first);
+        if (parentIndex >= 0) {
+            parentAbsPose = _skeleton->getAbsolutePose(parentIndex, _relativePoses);
+        }
+
+        // if parent should "look-at" child joint position.
+        if (shoulderShouldLookAtArm && (iter.first == leftArmIndex || iter.first == rightArmIndex)) {
+
+            AnimPose grandParentAbsPose;
+            int grandParentIndex = _skeleton->getParentIndex(parentIndex);
+            if (parentIndex >= 0) {
+                grandParentAbsPose = _skeleton->getAbsolutePose(grandParentIndex, _relativePoses);
+            }
+
+            // the shoulder should rotate toward the arm joint via "look-at" constraint
+            parentAbsPose = boneLookAt(absPose.trans(), parentAbsPose);
+            _relativePoses[parentIndex] = grandParentAbsPose.inverse() * parentAbsPose;
+        }
+
+        // Ignore translation on secondary poses, to prevent them from distorting the skeleton.
+        glm::vec3 origTrans = _relativePoses[iter.first].trans();
+        _relativePoses[iter.first] = parentAbsPose.inverse() * absPose;
+        _relativePoses[iter.first].trans() = origTrans;
+    }
+}
+
 void AnimInverseKinematics::initRelativePosesFromSolutionSource(SolutionSource solutionSource, const AnimPoseVec& underPoses) {
     const float RELAX_BLEND_FACTOR = (1.0f / 16.0f);
     const float COPY_BLEND_FACTOR = 1.0f;

libraries/animation/src/AnimInverseKinematics.h

@@ -70,6 +70,9 @@ public:
         NumSolutionSources,
     };
 
+    void setSecondaryTargetInRigFrame(int jointIndex, const AnimPose& pose);
+    void clearSecondaryTarget(int jointIndex);
+
     void setSolutionSource(SolutionSource solutionSource) { _solutionSource = solutionSource; }
     void setSolutionSourceVar(const QString& solutionSourceVar) { _solutionSourceVar = solutionSourceVar; }
 
@@ -88,6 +91,7 @@ protected:
     void initRelativePosesFromSolutionSource(SolutionSource solutionSource, const AnimPoseVec& underPose);
     void blendToPoses(const AnimPoseVec& targetPoses, const AnimPoseVec& underPose, float blendFactor);
     void preconditionRelativePosesToAvoidLimbLock(const AnimContext& context, const std::vector<IKTarget>& targets);
+    void setSecondaryTargets(const AnimContext& context);
     AnimPose applyHipsOffset() const;
 
     // used to pre-compute information about each joint influeced by a spline IK target.
@@ -142,6 +146,8 @@ protected:
     AnimPoseVec _relativePoses; // current relative poses
     AnimPoseVec _limitCenterPoses;  // relative
 
+    std::map<int, AnimPose> _secondaryTargetsInRigFrame;
+
     mutable std::map<int, std::vector<SplineJointInfo>> _splineJointInfoMap;
 
     // experimental data for moving hips during IK

libraries/animation/src/AnimUtil.cpp

@@ -96,3 +96,14 @@ float accumulateTime(float startFrame, float endFrame, float timeScale, float cu
     return frame;
 }
 
+// rotate bone's y-axis with target.
+AnimPose boneLookAt(const glm::vec3& target, const AnimPose& bone) {
+    glm::vec3 u, v, w;
+    generateBasisVectors(target - bone.trans(), bone.rot() * Vectors::UNIT_X, u, v, w);
+    glm::mat4 lookAt(glm::vec4(v, 0.0f),
+                     glm::vec4(u, 0.0f),
+                     // AJT: TODO REVISIT THIS, this could be -w.
+                     glm::vec4(glm::normalize(glm::cross(v, u)), 0.0f),
+                     glm::vec4(bone.trans(), 1.0f));
+    return AnimPose(lookAt);
+}

libraries/animation/src/AnimUtil.h

@@ -31,4 +31,6 @@ inline glm::quat safeLerp(const glm::quat& a, const glm::quat& b, float alpha) {
     return glm::normalize(glm::lerp(a, bTemp, alpha));
 }
 
+AnimPose boneLookAt(const glm::vec3& target, const AnimPose& bone);
+
 #endif

libraries/animation/src/Rig.cpp

@@ -1399,24 +1399,25 @@ void Rig::updateFromControllerParameters(const ControllerParameters& params, flo
     _animVars.set("isTalking", params.isTalking);
     _animVars.set("notIsTalking", !params.isTalking);
 
-    bool headEnabled = params.controllerActiveFlags[ControllerType_Head];
+    bool headEnabled = params.primaryControllerActiveFlags[PrimaryControllerType_Head];
-    bool leftHandEnabled = params.controllerActiveFlags[ControllerType_LeftHand];
+    bool leftHandEnabled = params.primaryControllerActiveFlags[PrimaryControllerType_LeftHand];
-    bool rightHandEnabled = params.controllerActiveFlags[ControllerType_RightHand];
+    bool rightHandEnabled = params.primaryControllerActiveFlags[PrimaryControllerType_RightHand];
-    bool hipsEnabled = params.controllerActiveFlags[ControllerType_Hips];
+    bool hipsEnabled = params.primaryControllerActiveFlags[PrimaryControllerType_Hips];
-    bool leftFootEnabled = params.controllerActiveFlags[ControllerType_LeftFoot];
+    bool leftFootEnabled = params.primaryControllerActiveFlags[PrimaryControllerType_LeftFoot];
-    bool rightFootEnabled = params.controllerActiveFlags[ControllerType_RightFoot];
+    bool rightFootEnabled = params.primaryControllerActiveFlags[PrimaryControllerType_RightFoot];
-    bool leftArmEnabled = params.controllerActiveFlags[ControllerType_LeftArm];
+    bool spine2Enabled = params.primaryControllerActiveFlags[PrimaryControllerType_Spine2];
-    bool rightArmEnabled = params.controllerActiveFlags[ControllerType_RightArm];
-    bool spine2Enabled = params.controllerActiveFlags[ControllerType_Spine2];
 
-    updateHead(headEnabled, hipsEnabled, params.controllerPoses[ControllerType_Head]);
+    bool leftArmEnabled = params.secondaryControllerActiveFlags[SecondaryControllerType_LeftArm];
+    bool rightArmEnabled = params.secondaryControllerActiveFlags[SecondaryControllerType_RightArm];
+
+    updateHead(headEnabled, hipsEnabled, params.primaryControllerPoses[PrimaryControllerType_Head]);
 
     updateHands(leftHandEnabled, rightHandEnabled, hipsEnabled, leftArmEnabled, rightArmEnabled, dt,
-                params.controllerPoses[ControllerType_LeftHand], params.controllerPoses[ControllerType_RightHand],
+                params.primaryControllerPoses[PrimaryControllerType_LeftHand], params.primaryControllerPoses[PrimaryControllerType_RightHand],
                 params.bodyCapsuleRadius, params.bodyCapsuleHalfHeight, params.bodyCapsuleLocalOffset);
 
     updateFeet(leftFootEnabled, rightFootEnabled,
-               params.controllerPoses[ControllerType_LeftFoot], params.controllerPoses[ControllerType_RightFoot]);
+               params.primaryControllerPoses[PrimaryControllerType_LeftFoot], params.primaryControllerPoses[PrimaryControllerType_RightFoot]);
 
     // if the hips or the feet are being controlled.
     if (hipsEnabled || rightFootEnabled || leftFootEnabled) {
@@ -1437,34 +1438,46 @@ void Rig::updateFromControllerParameters(const ControllerParameters& params, flo
 
     if (hipsEnabled) {
         _animVars.set("hipsType", (int)IKTarget::Type::RotationAndPosition);
-        _animVars.set("hipsPosition", params.controllerPoses[ControllerType_Hips].trans());
+        _animVars.set("hipsPosition", params.primaryControllerPoses[PrimaryControllerType_Hips].trans());
-        _animVars.set("hipsRotation", params.controllerPoses[ControllerType_Hips].rot());
+        _animVars.set("hipsRotation", params.primaryControllerPoses[PrimaryControllerType_Hips].rot());
     } else {
         _animVars.set("hipsType", (int)IKTarget::Type::Unknown);
     }
 
     if (hipsEnabled && spine2Enabled) {
         _animVars.set("spine2Type", (int)IKTarget::Type::Spline);
-        _animVars.set("spine2Position", params.controllerPoses[ControllerType_Spine2].trans());
+        _animVars.set("spine2Position", params.primaryControllerPoses[PrimaryControllerType_Spine2].trans());
-        _animVars.set("spine2Rotation", params.controllerPoses[ControllerType_Spine2].rot());
+        _animVars.set("spine2Rotation", params.primaryControllerPoses[PrimaryControllerType_Spine2].rot());
     } else {
         _animVars.set("spine2Type", (int)IKTarget::Type::Unknown);
     }
 
-    if (leftArmEnabled) {
+    // set secondary targets
-        _animVars.set("leftArmType", (int)IKTarget::Type::RotationAndPosition);
+    static const std::vector<QString> secondaryControllerJointNames = {
-        _animVars.set("leftArmPosition", params.controllerPoses[ControllerType_LeftArm].trans());
+        "LeftShoulder",
-        _animVars.set("leftArmRotation", params.controllerPoses[ControllerType_LeftArm].rot());
+        "RightShoulder",
-    } else {
+        "LeftArm",
-        _animVars.set("leftArmType", (int)IKTarget::Type::Unknown);
+        "RightArm",
-    }
+        "LeftForeArm",
+        "RightForeArm",
+        "LeftUpLeg",
+        "RightUpLeg",
+        "LeftLeg",
+        "RightLeg",
+        "LeftToeBase",
+        "RightToeBase"
+    };
 
-    if (rightArmEnabled) {
+    std::shared_ptr<AnimInverseKinematics> ikNode = getAnimInverseKinematicsNode();
-        _animVars.set("rightArmType", (int)IKTarget::Type::RotationAndPosition);
+    for (int i = 0; i < (int)NumSecondaryControllerTypes; i++) {
-        _animVars.set("rightArmPosition", params.controllerPoses[ControllerType_RightArm].trans());
+        int index = indexOfJoint(secondaryControllerJointNames[i]);
-        _animVars.set("rightArmRotation", params.controllerPoses[ControllerType_RightArm].rot());
+        if (index >= 0) {
-    } else {
+            if (params.secondaryControllerActiveFlags[i]) {
-        _animVars.set("rightArmType", (int)IKTarget::Type::Unknown);
+                ikNode->setSecondaryTargetInRigFrame(index, params.secondaryControllerPoses[i]);
+            } else {
+                ikNode->clearSecondaryTarget(index);
+            }
+        }
     }
 }
 

libraries/animation/src/Rig.h

@@ -41,22 +41,39 @@ public:
         bool useNames;
     };
 
-    enum ControllerType {
+    enum PrimaryControllerType {
-        ControllerType_Head = 0,
+        PrimaryControllerType_Head = 0,
-        ControllerType_LeftHand,
+        PrimaryControllerType_LeftHand,
-        ControllerType_RightHand,
+        PrimaryControllerType_RightHand,
-        ControllerType_Hips,
+        PrimaryControllerType_Hips,
-        ControllerType_LeftFoot,
+        PrimaryControllerType_LeftFoot,
-        ControllerType_RightFoot,
+        PrimaryControllerType_RightFoot,
-        ControllerType_LeftArm,
+        PrimaryControllerType_Spine2,
-        ControllerType_RightArm,
+        NumPrimaryControllerTypes
-        ControllerType_Spine2,
+    };
-        NumControllerTypes
+
+    // NOTE: These should ordered such that joint parents appear before their children.
+    enum SecondaryControllerType {
+        SecondaryControllerType_LeftShoulder = 0,
+        SecondaryControllerType_RightShoulder,
+        SecondaryControllerType_LeftArm,
+        SecondaryControllerType_RightArm,
+        SecondaryControllerType_LeftForeArm,
+        SecondaryControllerType_RightForeArm,
+        SecondaryControllerType_LeftUpLeg,
+        SecondaryControllerType_RightUpLeg,
+        SecondaryControllerType_LeftLeg,
+        SecondaryControllerType_RightLeg,
+        SecondaryControllerType_LeftToeBase,
+        SecondaryControllerType_RightToeBase,
+        NumSecondaryControllerTypes
     };
 
     struct ControllerParameters {
-        AnimPose controllerPoses[NumControllerTypes];  // rig space
+        AnimPose primaryControllerPoses[NumPrimaryControllerTypes];  // rig space
-        bool controllerActiveFlags[NumControllerTypes];
+        bool primaryControllerActiveFlags[NumPrimaryControllerTypes];
+        AnimPose secondaryControllerPoses[NumSecondaryControllerTypes];  // rig space
+        bool secondaryControllerActiveFlags[NumSecondaryControllerTypes];
         bool isTalking;
         float bodyCapsuleRadius;
         float bodyCapsuleHalfHeight;

libraries/audio-client/src/AudioClient.h

@@ -364,7 +364,7 @@ private:
     AudioIOStats _stats;
 
     AudioGate* _audioGate { nullptr };
-    bool _audioGateOpen { false };
+    bool _audioGateOpen { true };
 
     AudioPositionGetter _positionGetter;
     AudioOrientationGetter _orientationGetter;

libraries/avatars-renderer/src/avatars-renderer/Avatar.cpp

@@ -151,11 +151,6 @@ glm::vec3 Avatar::getNeckPosition() const {
     return _skeletonModel->getNeckPosition(neckPosition) ? neckPosition : getPosition();
 }
 
-
-glm::quat Avatar::getWorldAlignedOrientation () const {
-    return computeRotationFromBodyToWorldUp() * getOrientation();
-}
-
 AABox Avatar::getBounds() const {
     if (!_skeletonModel->isRenderable() || _skeletonModel->needsFixupInScene()) {
         // approximately 2m tall, scaled to user request.
@@ -436,6 +431,11 @@ void Avatar::slamPosition(const glm::vec3& newPosition) {
     _lastVelocity = glm::vec3(0.0f);
 }
 
+void Avatar::updateAttitude(const glm::quat& orientation) {
+    _skeletonModel->updateAttitude(orientation);
+	_worldUpDirection = orientation * Vectors::UNIT_Y;
+}
+
 void Avatar::applyPositionDelta(const glm::vec3& delta) {
     setPosition(getPosition() + delta);
     _positionDeltaAccumulator += delta;
@@ -628,22 +628,6 @@ void Avatar::render(RenderArgs* renderArgs) {
     }
 }
 
-glm::quat Avatar::computeRotationFromBodyToWorldUp(float proportion) const {
-    glm::quat orientation = getOrientation();
-    glm::vec3 currentUp = orientation * IDENTITY_UP;
-    float angle = acosf(glm::clamp(glm::dot(currentUp, _worldUpDirection), -1.0f, 1.0f));
-    if (angle < EPSILON) {
-        return glm::quat();
-    }
-    glm::vec3 axis;
-    if (angle > 179.99f * RADIANS_PER_DEGREE) { // 180 degree rotation; must use another axis
-        axis = orientation * IDENTITY_RIGHT;
-    } else {
-        axis = glm::normalize(glm::cross(currentUp, _worldUpDirection));
-    }
-    return glm::angleAxis(angle * proportion, axis);
-}
-
 void Avatar::fixupModelsInScene(const render::ScenePointer& scene) {
     _attachmentsToDelete.clear();
 
@@ -915,17 +899,34 @@ glm::vec3 Avatar::getDefaultJointTranslation(int index) const {
 }
 
 glm::quat Avatar::getAbsoluteDefaultJointRotationInObjectFrame(int index) const {
-    glm::quat rotation;
+    // To make this thread safe, we hold onto the model by smart ptr, which prevents it from being deleted while we are accessing it.
-    glm::quat rot = _skeletonModel->getRig().getAnimSkeleton()->getAbsoluteDefaultPose(index).rot();
+    auto model = getSkeletonModel();
-    return Quaternions::Y_180 * rot;
+    if (model) {
+        auto skeleton = model->getRig().getAnimSkeleton();
+        if (skeleton && index >= 0 && index < skeleton->getNumJoints()) {
+            // The rotation part of the geometry-to-rig transform is always identity so we can skip it.
+            // Y_180 is to convert from rig-frame into avatar-frame
+            return Quaternions::Y_180 * skeleton->getAbsoluteDefaultPose(index).rot();
+        }
+    }
+    return Quaternions::Y_180;
 }
 
 glm::vec3 Avatar::getAbsoluteDefaultJointTranslationInObjectFrame(int index) const {
-    glm::vec3 translation;
+    // To make this thread safe, we hold onto the model by smart ptr, which prevents it from being deleted while we are accessing it.
-    const Rig& rig = _skeletonModel->getRig();
+    auto model = getSkeletonModel();
-    glm::vec3 trans = rig.getAnimSkeleton()->getAbsoluteDefaultPose(index).trans();
+    if (model) {
-    glm::mat4 y180Mat = createMatFromQuatAndPos(Quaternions::Y_180, glm::vec3());
+        const Rig& rig = model->getRig();
-    return transformPoint(y180Mat * rig.getGeometryToRigTransform(), trans);
+        auto skeleton = rig.getAnimSkeleton();
+        if (skeleton && index >= 0 && index < skeleton->getNumJoints()) {
+            // trans is in geometry frame.
+            glm::vec3 trans = skeleton->getAbsoluteDefaultPose(index).trans();
+            // Y_180 is to convert from rig-frame into avatar-frame
+            glm::mat4 geomToAvatarMat = Matrices::Y_180 * rig.getGeometryToRigTransform();
+            return transformPoint(geomToAvatarMat, trans);
+        }
+    }
+    return Vectors::ZERO;
 }
 
 glm::quat Avatar::getAbsoluteJointRotationInObjectFrame(int index) const {
@@ -1401,14 +1402,14 @@ glm::quat Avatar::getUncachedRightPalmRotation() const {
     return rightPalmRotation;
 }
 
-void Avatar::setPosition(const glm::vec3& position) {
+void Avatar::setPositionViaScript(const glm::vec3& position) {
-    AvatarData::setPosition(position);
+    setPosition(position);
-    updateAttitude();
+    updateAttitude(getOrientation());
 }
 
-void Avatar::setOrientation(const glm::quat& orientation) {
+void Avatar::setOrientationViaScript(const glm::quat& orientation) {
-    AvatarData::setOrientation(orientation);
+    setOrientation(orientation);
-    updateAttitude();
+    updateAttitude(orientation);
 }
 
 void Avatar::updatePalms() {

libraries/avatars-renderer/src/avatars-renderer/Avatar.h

@@ -112,8 +112,6 @@ public:
     const Head* getHead() const { return static_cast<const Head*>(_headData); }
     Head* getHead() { return static_cast<Head*>(_headData); }
 
-    glm::quat getWorldAlignedOrientation() const;
-
     AABox getBounds() const;
 
     /// Returns the distance to use as a LOD parameter.
@@ -184,7 +182,7 @@ public:
     void scaleVectorRelativeToPosition(glm::vec3 &positionToScale) const;
 
     void slamPosition(const glm::vec3& position);
-    virtual void updateAttitude() override { _skeletonModel->updateAttitude(); }
+    virtual void updateAttitude(const glm::quat& orientation) override;
 
     // Call this when updating Avatar position with a delta.  This will allow us to
     // _accurately_ measure position changes and compute the resulting velocity
@@ -197,10 +195,8 @@ public:
     void getCapsule(glm::vec3& start, glm::vec3& end, float& radius);
     float computeMass();
 
-    using SpatiallyNestable::setPosition;
+    void setPositionViaScript(const glm::vec3& position) override;
-    virtual void setPosition(const glm::vec3& position) override;
+    void setOrientationViaScript(const glm::quat& orientation) override;
-    using SpatiallyNestable::setOrientation;
-    virtual void setOrientation(const glm::quat& orientation) override;
 
     // these call through to the SpatiallyNestable versions, but they are here to expose these to javascript.
     Q_INVOKABLE virtual const QUuid getParentID() const override { return SpatiallyNestable::getParentID(); }
@@ -240,7 +236,7 @@ public:
     bool hasNewJointData() const { return _hasNewJointData; }
 
     float getBoundingRadius() const;
-    
+
     void addToScene(AvatarSharedPointer self, const render::ScenePointer& scene);
     void ensureInScene(AvatarSharedPointer self, const render::ScenePointer& scene);
     bool isInScene() const { return render::Item::isValidID(_renderItemID); }
@@ -303,7 +299,6 @@ protected:
 
     glm::vec3 getBodyRightDirection() const { return getOrientation() * IDENTITY_RIGHT; }
     glm::vec3 getBodyUpDirection() const { return getOrientation() * IDENTITY_UP; }
-    glm::quat computeRotationFromBodyToWorldUp(float proportion = 1.0f) const;
     void measureMotionDerivatives(float deltaTime);
 
     float getSkeletonHeight() const;

libraries/avatars-renderer/src/avatars-renderer/SkeletonModel.cpp

@@ -118,16 +118,16 @@ void SkeletonModel::updateRig(float deltaTime, glm::mat4 parentTransform) {
     _rig.updateFromEyeParameters(eyeParams);
 }
 
-void SkeletonModel::updateAttitude() {
+void SkeletonModel::updateAttitude(const glm::quat& orientation) {
     setTranslation(_owningAvatar->getSkeletonPosition());
-    setRotation(_owningAvatar->getOrientation() * Quaternions::Y_180);
+    setRotation(orientation * Quaternions::Y_180);
     setScale(glm::vec3(1.0f, 1.0f, 1.0f) * _owningAvatar->getScale());
 }
 
 // Called by Avatar::simulate after it has set the joint states (fullUpdate true if changed),
 // but just before head has been simulated.
 void SkeletonModel::simulate(float deltaTime, bool fullUpdate) {
-    updateAttitude();
+    updateAttitude(_owningAvatar->getOrientation());
     if (fullUpdate) {
         setBlendshapeCoefficients(_owningAvatar->getHead()->getSummedBlendshapeCoefficients());
 

libraries/avatars-renderer/src/avatars-renderer/SkeletonModel.h

@@ -35,7 +35,7 @@ public:
 
     void simulate(float deltaTime, bool fullUpdate = true) override;
     void updateRig(float deltaTime, glm::mat4 parentTransform) override;
-    void updateAttitude();
+    void updateAttitude(const glm::quat& orientation);
 
     /// Returns the index of the left hand joint, or -1 if not found.
     int getLeftHandJointIndex() const { return isActive() ? getFBXGeometry().leftHandJointIndex : -1; }

libraries/avatars/src/AvatarData.cpp

@@ -91,9 +91,6 @@ AvatarData::AvatarData() :
     _targetVelocity(0.0f),
     _density(DEFAULT_AVATAR_DENSITY)
 {
-    setBodyPitch(0.0f);
-    setBodyYaw(-90.0f);
-    setBodyRoll(0.0f);
 }
 
 AvatarData::~AvatarData() {
@@ -110,23 +107,6 @@ const QUrl& AvatarData::defaultFullAvatarModelUrl() {
     return _defaultFullAvatarModelUrl;
 }
 
-// There are a number of possible strategies for this set of tools through endRender, below.
-void AvatarData::nextAttitude(glm::vec3 position, glm::quat orientation) {
-    bool success;
-    Transform trans = getTransform(success);
-    if (!success) {
-        qCWarning(avatars) << "Warning -- AvatarData::nextAttitude failed";
-        return;
-    }
-    trans.setTranslation(position);
-    trans.setRotation(orientation);
-    SpatiallyNestable::setTransform(trans, success);
-    if (!success) {
-        qCWarning(avatars) << "Warning -- AvatarData::nextAttitude failed";
-    }
-    updateAttitude();
-}
-
 void AvatarData::setTargetScale(float targetScale) {
     auto newValue = glm::clamp(targetScale, MIN_AVATAR_SCALE, MAX_AVATAR_SCALE);
     if (_targetScale != newValue) {
@@ -2100,6 +2080,7 @@ void AvatarData::fromJson(const QJsonObject& json, bool useFrameSkeleton) {
         currentBasis = std::make_shared<Transform>(Transform::fromJson(json[JSON_AVATAR_BASIS]));
     }
 
+    glm::quat orientation;
     if (json.contains(JSON_AVATAR_RELATIVE)) {
         // During playback you can either have the recording basis set to the avatar current state
         // meaning that all playback is relative to this avatars starting position, or
@@ -2111,12 +2092,14 @@ void AvatarData::fromJson(const QJsonObject& json, bool useFrameSkeleton) {
         auto relativeTransform = Transform::fromJson(json[JSON_AVATAR_RELATIVE]);
         auto worldTransform = currentBasis->worldTransform(relativeTransform);
         setPosition(worldTransform.getTranslation());
-        setOrientation(worldTransform.getRotation());
+        orientation = worldTransform.getRotation();
     } else {
         // We still set the position in the case that there is no movement.
         setPosition(currentBasis->getTranslation());
-        setOrientation(currentBasis->getRotation());
+        orientation = currentBasis->getRotation();
     }
+    setOrientation(orientation);
+    updateAttitude(orientation);
 
     // Do after avatar orientation because head look-at needs avatar orientation.
     if (json.contains(JSON_AVATAR_HEAD)) {
@@ -2234,11 +2217,11 @@ void AvatarData::setBodyRoll(float bodyRoll) {
     setOrientation(glm::quat(glm::radians(eulerAngles)));
 }
 
-void AvatarData::setPosition(const glm::vec3& position) {
+void AvatarData::setPositionViaScript(const glm::vec3& position) {
     SpatiallyNestable::setPosition(position);
 }
 
-void AvatarData::setOrientation(const glm::quat& orientation) {
+void AvatarData::setOrientationViaScript(const glm::quat& orientation) {
     SpatiallyNestable::setOrientation(orientation);
 }
 

libraries/avatars/src/AvatarData.h

@@ -351,14 +351,14 @@ public:
 class AvatarData : public QObject, public SpatiallyNestable {
     Q_OBJECT
 
-    Q_PROPERTY(glm::vec3 position READ getPosition WRITE setPosition)
+    Q_PROPERTY(glm::vec3 position READ getPosition WRITE setPositionViaScript)
     Q_PROPERTY(float scale READ getTargetScale WRITE setTargetScale)
     Q_PROPERTY(glm::vec3 handPosition READ getHandPosition WRITE setHandPosition)
     Q_PROPERTY(float bodyYaw READ getBodyYaw WRITE setBodyYaw)
     Q_PROPERTY(float bodyPitch READ getBodyPitch WRITE setBodyPitch)
     Q_PROPERTY(float bodyRoll READ getBodyRoll WRITE setBodyRoll)
 
-    Q_PROPERTY(glm::quat orientation READ getOrientation WRITE setOrientation)
+    Q_PROPERTY(glm::quat orientation READ getOrientation WRITE setOrientationViaScript)
     Q_PROPERTY(glm::quat headOrientation READ getHeadOrientation WRITE setHeadOrientation)
     Q_PROPERTY(float headPitch READ getHeadPitch WRITE setHeadPitch)
     Q_PROPERTY(float headYaw READ getHeadYaw WRITE setHeadYaw)
@@ -440,13 +440,10 @@ public:
     float getBodyRoll() const;
     void setBodyRoll(float bodyRoll);
 
-    using SpatiallyNestable::setPosition;
+    virtual void setPositionViaScript(const glm::vec3& position);
-    virtual void setPosition(const glm::vec3& position) override;
+    virtual void setOrientationViaScript(const glm::quat& orientation);
-    using SpatiallyNestable::setOrientation;
-    virtual void setOrientation(const glm::quat& orientation) override;
 
-    void nextAttitude(glm::vec3 position, glm::quat orientation); // Can be safely called at any time.
+    virtual void updateAttitude(const glm::quat& orientation) {}
-    virtual void updateAttitude() {} // Tell skeleton mesh about changes
 
     glm::quat getHeadOrientation() const {
         lazyInitHeadData();

libraries/controllers/src/controllers/Actions.cpp

@@ -101,6 +101,23 @@ namespace controller {
             makePosePair(Action::RIGHT_HAND_PINKY3, "RightHandPinky3"),
             makePosePair(Action::RIGHT_HAND_PINKY4, "RightHandPinky4"),
 
+            makePosePair(Action::TRACKED_OBJECT_00, "TrackedObject00"),
+            makePosePair(Action::TRACKED_OBJECT_01, "TrackedObject01"),
+            makePosePair(Action::TRACKED_OBJECT_02, "TrackedObject02"),
+            makePosePair(Action::TRACKED_OBJECT_03, "TrackedObject03"),
+            makePosePair(Action::TRACKED_OBJECT_04, "TrackedObject04"),
+            makePosePair(Action::TRACKED_OBJECT_05, "TrackedObject05"),
+            makePosePair(Action::TRACKED_OBJECT_06, "TrackedObject06"),
+            makePosePair(Action::TRACKED_OBJECT_07, "TrackedObject07"),
+            makePosePair(Action::TRACKED_OBJECT_08, "TrackedObject08"),
+            makePosePair(Action::TRACKED_OBJECT_09, "TrackedObject09"),
+            makePosePair(Action::TRACKED_OBJECT_10, "TrackedObject10"),
+            makePosePair(Action::TRACKED_OBJECT_11, "TrackedObject11"),
+            makePosePair(Action::TRACKED_OBJECT_12, "TrackedObject12"),
+            makePosePair(Action::TRACKED_OBJECT_13, "TrackedObject13"),
+            makePosePair(Action::TRACKED_OBJECT_14, "TrackedObject14"),
+            makePosePair(Action::TRACKED_OBJECT_15, "TrackedObject15"),
+
             makeButtonPair(Action::LEFT_HAND_CLICK, "LeftHandClick"),
             makeButtonPair(Action::RIGHT_HAND_CLICK, "RightHandClick"),
 

libraries/controllers/src/controllers/Actions.h

@@ -101,6 +101,7 @@ enum class Action {
     // Bisected aliases for TRANSLATE_CAMERA_Z
     BOOM_IN,
     BOOM_OUT,
+
     LEFT_ARM,
     RIGHT_ARM,
 
@@ -146,7 +147,35 @@ enum class Action {
     RIGHT_HAND_PINKY3,
     RIGHT_HAND_PINKY4,
 
-    NUM_ACTIONS,
+    LEFT_SHOULDER,
+    RIGHT_SHOULDER,
+    LEFT_FORE_ARM,
+    RIGHT_FORE_ARM,
+    LEFT_LEG,
+    RIGHT_LEG,
+    LEFT_UP_LEG,
+    RIGHT_UP_LEG,
+    LEFT_TOE_BASE,
+    RIGHT_TOE_BASE,
+
+    TRACKED_OBJECT_00,
+    TRACKED_OBJECT_01,
+    TRACKED_OBJECT_02,
+    TRACKED_OBJECT_03,
+    TRACKED_OBJECT_04,
+    TRACKED_OBJECT_05,
+    TRACKED_OBJECT_06,
+    TRACKED_OBJECT_07,
+    TRACKED_OBJECT_08,
+    TRACKED_OBJECT_09,
+    TRACKED_OBJECT_10,
+    TRACKED_OBJECT_11,
+    TRACKED_OBJECT_12,
+    TRACKED_OBJECT_13,
+    TRACKED_OBJECT_14,
+    TRACKED_OBJECT_15,
+
+    NUM_ACTIONS
 };
 
 template <typename T>

libraries/controllers/src/controllers/InputDevice.cpp

@@ -15,20 +15,6 @@
 
 namespace controller {
 
-    const float DEFAULT_HAND_RETICLE_MOVE_SPEED = 37.5f;
-    float InputDevice::_reticleMoveSpeed = DEFAULT_HAND_RETICLE_MOVE_SPEED;
-
-    //Constants for getCursorPixelRangeMultiplier()
-    const float MIN_PIXEL_RANGE_MULT = 0.4f;
-    const float MAX_PIXEL_RANGE_MULT = 2.0f;
-    const float RANGE_MULT = (MAX_PIXEL_RANGE_MULT - MIN_PIXEL_RANGE_MULT) * 0.01f;
-
-    //Returns a multiplier to be applied to the cursor range for the controllers
-    float InputDevice::getCursorPixelRangeMult() {
-        //scales (0,100) to (MINIMUM_PIXEL_RANGE_MULT, MAXIMUM_PIXEL_RANGE_MULT)
-        return InputDevice::_reticleMoveSpeed * RANGE_MULT + MIN_PIXEL_RANGE_MULT;
-    }
-
     float InputDevice::getButton(int channel) const {
         if (!_buttonPressedMap.empty()) {
             if (_buttonPressedMap.find(channel) != _buttonPressedMap.end()) {

libraries/controllers/src/controllers/InputDevice.h

@@ -73,10 +73,6 @@ public:
     int getDeviceID() { return _deviceID; }
     void setDeviceID(int deviceID) { _deviceID = deviceID; }
 
-    static float getCursorPixelRangeMult();
-    static float getReticleMoveSpeed() { return _reticleMoveSpeed; }
-    static void setReticleMoveSpeed(float reticleMoveSpeed) { _reticleMoveSpeed = reticleMoveSpeed; }
-
     Input makeInput(StandardButtonChannel button) const;
     Input makeInput(StandardAxisChannel axis) const;
     Input makeInput(StandardPoseChannel pose) const;
@@ -99,9 +95,6 @@ protected:
     ButtonPressedMap _buttonPressedMap;
     AxisStateMap _axisStateMap;
     PoseStateMap _poseStateMap;
-
-private:
-    static float _reticleMoveSpeed;
 };
 
 }

libraries/controllers/src/controllers/StandardController.cpp

@@ -166,6 +166,23 @@ Input::NamedVector StandardController::getAvailableInputs() const {
         makePair(DD, "Down"),
         makePair(DL, "Left"),
         makePair(DR, "Right"),
+
+        makePair(TRACKED_OBJECT_00, "TrackedObject00"),
+        makePair(TRACKED_OBJECT_01, "TrackedObject01"),
+        makePair(TRACKED_OBJECT_02, "TrackedObject02"),
+        makePair(TRACKED_OBJECT_03, "TrackedObject03"),
+        makePair(TRACKED_OBJECT_04, "TrackedObject04"),
+        makePair(TRACKED_OBJECT_05, "TrackedObject05"),
+        makePair(TRACKED_OBJECT_06, "TrackedObject06"),
+        makePair(TRACKED_OBJECT_07, "TrackedObject07"),
+        makePair(TRACKED_OBJECT_08, "TrackedObject08"),
+        makePair(TRACKED_OBJECT_09, "TrackedObject09"),
+        makePair(TRACKED_OBJECT_10, "TrackedObject10"),
+        makePair(TRACKED_OBJECT_11, "TrackedObject11"),
+        makePair(TRACKED_OBJECT_12, "TrackedObject12"),
+        makePair(TRACKED_OBJECT_13, "TrackedObject13"),
+        makePair(TRACKED_OBJECT_14, "TrackedObject14"),
+        makePair(TRACKED_OBJECT_15, "TrackedObject15")
     };
     return availableInputs;
 }

libraries/controllers/src/controllers/impl/Filter.cpp

@@ -22,6 +22,7 @@
 #include "filters/DeadZoneFilter.h"
 #include "filters/HysteresisFilter.h"
 #include "filters/InvertFilter.h"
+#include "filters/NotFilter.h"
 #include "filters/PulseFilter.h"
 #include "filters/ScaleFilter.h"
 #include "filters/TranslateFilter.h"
@@ -40,6 +41,7 @@ REGISTER_FILTER_CLASS_INSTANCE(ConstrainToPositiveIntegerFilter, "constrainToPos
 REGISTER_FILTER_CLASS_INSTANCE(DeadZoneFilter, "deadZone")
 REGISTER_FILTER_CLASS_INSTANCE(HysteresisFilter, "hysteresis")
 REGISTER_FILTER_CLASS_INSTANCE(InvertFilter, "invert")
+REGISTER_FILTER_CLASS_INSTANCE(NotFilter, "logicalNot")
 REGISTER_FILTER_CLASS_INSTANCE(ScaleFilter, "scale")
 REGISTER_FILTER_CLASS_INSTANCE(PulseFilter, "pulse")
 REGISTER_FILTER_CLASS_INSTANCE(TranslateFilter, "translate")

libraries/controllers/src/controllers/impl/RouteBuilderProxy.cpp

@@ -24,6 +24,7 @@
 #include "filters/DeadZoneFilter.h"
 #include "filters/HysteresisFilter.h"
 #include "filters/InvertFilter.h"
+#include "filters/NotFilter.h"
 #include "filters/PulseFilter.h"
 #include "filters/ScaleFilter.h"
 #include "filters/TranslateFilter.h"
@@ -148,6 +149,11 @@ QObject* RouteBuilderProxy::pulse(float interval) {
     return this;
 }
 
+QObject* RouteBuilderProxy::logicalNot() {
+    addFilter(std::make_shared<NotFilter>());
+    return this;
+}
+
 void RouteBuilderProxy::addFilter(Filter::Pointer filter) {
     _route->filters.push_back(filter);
 }

libraries/controllers/src/controllers/impl/RouteBuilderProxy.h

@@ -53,6 +53,7 @@ class RouteBuilderProxy : public QObject {
         Q_INVOKABLE QObject* postTransform(glm::mat4 transform);
         Q_INVOKABLE QObject* rotate(glm::quat rotation);
         Q_INVOKABLE QObject* lowVelocity(float rotationConstant, float translationConstant);
+        Q_INVOKABLE QObject* logicalNot();
 
     private:
         void to(const Endpoint::Pointer& destination);

libraries/controllers/src/controllers/impl/filters/NotFilter.cpp

@@ -0,0 +1,10 @@
+#include "NotFilter.h"
+
+using namespace controller;
+
+NotFilter::NotFilter() {
+}
+
+float NotFilter::apply(float value) const {
+    return (value == 0.0f) ? 1.0f : 0.0f;
+}

libraries/controllers/src/controllers/impl/filters/NotFilter.h

@@ -0,0 +1,21 @@
+#pragma once
+#ifndef hifi_Controllers_Filters_Not_h
+#define hifi_Controllers_Filters_Not_h
+
+#include "../Filter.h"
+
+namespace controller {
+
+class NotFilter : public Filter {
+    REGISTER_FILTER_CLASS(NotFilter);
+public:
+    NotFilter();
+
+    virtual float apply(float value) const override;
+    virtual Pose apply(Pose value) const override { return value; }
+};
+
+}
+
+#endif
+

libraries/entities-renderer/src/EntityTreeRenderer.cpp

@@ -26,7 +26,6 @@
 #include <PerfStat.h>
 #include <SceneScriptingInterface.h>
 #include <ScriptEngine.h>
-#include <HoverOverlayInterface.h>
 
 
 #include "RenderableEntityItem.h"
@@ -453,8 +452,6 @@ RayToEntityIntersectionResult EntityTreeRenderer::findRayIntersectionWorker(cons
 
 void EntityTreeRenderer::connectSignalsToSlots(EntityScriptingInterface* entityScriptingInterface) {
 
-    auto hoverOverlayInterface = DependencyManager::get<HoverOverlayInterface>().data();
-
     connect(this, &EntityTreeRenderer::mousePressOnEntity, entityScriptingInterface, &EntityScriptingInterface::mousePressOnEntity);
     connect(this, &EntityTreeRenderer::mouseMoveOnEntity, entityScriptingInterface, &EntityScriptingInterface::mouseMoveOnEntity);
     connect(this, &EntityTreeRenderer::mouseReleaseOnEntity, entityScriptingInterface, &EntityScriptingInterface::mouseReleaseOnEntity);
@@ -464,12 +461,8 @@ void EntityTreeRenderer::connectSignalsToSlots(EntityScriptingInterface* entityS
     connect(this, &EntityTreeRenderer::clickReleaseOnEntity, entityScriptingInterface, &EntityScriptingInterface::clickReleaseOnEntity);
 
     connect(this, &EntityTreeRenderer::hoverEnterEntity, entityScriptingInterface, &EntityScriptingInterface::hoverEnterEntity);
-    connect(this, SIGNAL(hoverEnterEntity(const EntityItemID&, const PointerEvent&)), hoverOverlayInterface, SLOT(createHoverOverlay(const EntityItemID&, const PointerEvent&)));
-
     connect(this, &EntityTreeRenderer::hoverOverEntity, entityScriptingInterface, &EntityScriptingInterface::hoverOverEntity);
-
     connect(this, &EntityTreeRenderer::hoverLeaveEntity, entityScriptingInterface, &EntityScriptingInterface::hoverLeaveEntity);
-    connect(this, SIGNAL(hoverLeaveEntity(const EntityItemID&, const PointerEvent&)), hoverOverlayInterface, SLOT(destroyHoverOverlay(const EntityItemID&, const PointerEvent&)));
 
     connect(this, &EntityTreeRenderer::enterEntity, entityScriptingInterface, &EntityScriptingInterface::enterEntity);
     connect(this, &EntityTreeRenderer::leaveEntity, entityScriptingInterface, &EntityScriptingInterface::leaveEntity);
@@ -682,7 +675,7 @@ void EntityTreeRenderer::mouseMoveEvent(QMouseEvent* event) {
 
     PickRay ray = _viewState->computePickRay(event->x(), event->y());
 
-    bool precisionPicking = false; // for mouse moves we do not do precision picking
+    bool precisionPicking = true; // for mouse moves we do precision picking
     RayToEntityIntersectionResult rayPickResult = findRayIntersectionWorker(ray, Octree::TryLock, precisionPicking);
     if (rayPickResult.intersects) {
 

libraries/entities-renderer/src/RenderableModelEntityItem.cpp

@@ -372,6 +372,21 @@ void RenderableModelEntityItem::render(RenderArgs* args) {
         _model->updateRenderItems();
     }
 
+    // this simple logic should say we set showingEntityHighlight to true whenever we are in marketplace mode and we have a marketplace id, or
+    // whenever we are not set to none and shouldHighlight is true.
+    bool showingEntityHighlight = ((bool)(args->_outlineFlags & (int)RenderArgs::RENDER_OUTLINE_MARKETPLACE_MODE) && getMarketplaceID().length() != 0) ||
+                                  (args->_outlineFlags != RenderArgs::RENDER_OUTLINE_NONE && getShouldHighlight());
+    if (showingEntityHighlight) {
+        static glm::vec4 yellowColor(1.0f, 1.0f, 0.0f, 1.0f);
+        gpu::Batch& batch = *args->_batch;
+        bool success;
+        auto shapeTransform = getTransformToCenter(success);
+        if (success) {
+            batch.setModelTransform(shapeTransform); // we want to include the scale as well
+            DependencyManager::get<GeometryCache>()->renderWireCubeInstance(args, batch, yellowColor);
+        }
+    }
+
     if (!hasModel() || (_model && _model->didVisualGeometryRequestFail())) {
         static glm::vec4 greenColor(0.0f, 1.0f, 0.0f, 1.0f);
         gpu::Batch& batch = *args->_batch;

libraries/entities-renderer/src/RenderableShapeEntityItem.cpp

@@ -82,6 +82,80 @@ bool RenderableShapeEntityItem::isTransparent() {
     }
 }
 
+void RenderableShapeEntityItem::computeShapeInfo(ShapeInfo& info) {
+
+    // This will be called whenever DIRTY_SHAPE flag (set by dimension change, etc)
+    // is set.
+
+    const glm::vec3 entityDimensions = getDimensions();
+
+    switch (_shape){
+        case entity::Shape::Quad:
+        case entity::Shape::Cube: {
+            _collisionShapeType = SHAPE_TYPE_BOX;
+        }
+        break;
+        case entity::Shape::Sphere: {
+
+            float diameter = entityDimensions.x;
+            const float MIN_DIAMETER = 0.001f;
+            const float MIN_RELATIVE_SPHERICAL_ERROR = 0.001f;
+            if (diameter > MIN_DIAMETER
+                && fabsf(diameter - entityDimensions.y) / diameter < MIN_RELATIVE_SPHERICAL_ERROR
+                && fabsf(diameter - entityDimensions.z) / diameter < MIN_RELATIVE_SPHERICAL_ERROR) {
+
+                _collisionShapeType = SHAPE_TYPE_SPHERE;
+            }
+            else {
+                _collisionShapeType = SHAPE_TYPE_ELLIPSOID;
+            }
+        }
+        break;
+        case entity::Shape::Cylinder: {
+            _collisionShapeType = SHAPE_TYPE_CYLINDER_Y;
+            // TODO WL21389: determine if rotation is axis-aligned
+            //const Transform::Quat & rot = _transform.getRotation();
+
+            // TODO WL21389: some way to tell apart SHAPE_TYPE_CYLINDER_Y, _X, _Z based on rotation and
+            //       hull ( or dimensions, need circular cross section)
+            // Should allow for minor variance along axes?
+
+        }
+        break;
+        case entity::Shape::Triangle:
+        case entity::Shape::Hexagon:
+        case entity::Shape::Octagon:
+        case entity::Shape::Circle:
+        case entity::Shape::Tetrahedron:
+        case entity::Shape::Octahedron:
+        case entity::Shape::Dodecahedron:
+        case entity::Shape::Icosahedron:
+        case entity::Shape::Cone: {
+            //TODO WL21389: SHAPE_TYPE_SIMPLE_HULL and pointCollection (later)
+            _collisionShapeType = SHAPE_TYPE_ELLIPSOID;
+        }
+        break;
+        case entity::Shape::Torus:
+        {
+            // Not in GeometryCache::buildShapes, unsupported.
+            _collisionShapeType = SHAPE_TYPE_ELLIPSOID;
+            //TODO WL21389: SHAPE_TYPE_SIMPLE_HULL and pointCollection (later if desired support)
+        }
+        break;
+        default:{
+            _collisionShapeType = SHAPE_TYPE_ELLIPSOID;
+        }
+        break;
+    }
+
+    EntityItem::computeShapeInfo(info);
+}
+
+// This value specifes how the shape should be treated by physics calculations.
+ShapeType RenderableShapeEntityItem::getShapeType() const {
+    return _collisionShapeType;
+}
+
 void RenderableShapeEntityItem::render(RenderArgs* args) {
     PerformanceTimer perfTimer("RenderableShapeEntityItem::render");
     //Q_ASSERT(getType() == EntityTypes::Shape);

libraries/entities-renderer/src/RenderableShapeEntityItem.h

@@ -28,9 +28,18 @@ public:
 
     bool isTransparent() override;
 
+    virtual void computeShapeInfo(ShapeInfo& info) override;
+    ShapeType getShapeType() const override;
+
+
 private:
     std::unique_ptr<Procedural> _procedural { nullptr };
 
+    //! This is SHAPE_TYPE_ELLIPSOID rather than SHAPE_TYPE_NONE to maintain
+    //! prior functionality where new or unsupported shapes are treated as
+    //! ellipsoids.
+    ShapeType _collisionShapeType{ ShapeType::SHAPE_TYPE_ELLIPSOID };
+
     SIMPLE_RENDERABLE();
 };