diff --git a/tools/auto-tester/Create.PNG b/tools/auto-tester/Create.PNG index d82d4873a2..4a2a77d2f8 100644 Binary files a/tools/auto-tester/Create.PNG and b/tools/auto-tester/Create.PNG differ diff --git a/tools/auto-tester/Evaluate.PNG b/tools/auto-tester/Evaluate.PNG index d530dec994..15f5bb47b0 100644 Binary files a/tools/auto-tester/Evaluate.PNG and b/tools/auto-tester/Evaluate.PNG differ diff --git a/tools/auto-tester/README.md b/tools/auto-tester/README.md index 0924e77f8b..9463df4b43 100644 --- a/tools/auto-tester/README.md +++ b/tools/auto-tester/README.md @@ -36,6 +36,9 @@ This function is used to create/update Expected Images after a successful run of The user will be asked for the snapshot folder and then the tests root folder. All snapshots located in the snapshot folder will be used to create or update the expected images in the relevant tests. ### Details As an example - if the snapshots folder contains an image named `tests.content.entity.zone.zoneOrientation.00003.png`, then this file will be copied to `tests/contente/enity/zone/zoneOrientation/ExpectedImage0003.png`. +## Create Tests Outline +### Usage +This function creates an MD file in the (user-selected) tests root folder. The file provides links to both the tests and the MD files. ## Create MD file ### Usage This function creates a file named `test.md` from a `test.js` script. The user will be asked for the folder containing the test script: @@ -54,13 +57,20 @@ The process to produce the MD file is a simplistic parse of the test script. ### Usage This function creates all MD files recursively from the user-selected root folder. This can be any folder in the tests hierarchy (e.g. all engine\material tests). -The file provides a hierarchial list of all the tests -## Create Tests Outline +The file provides a hierarchal list of all the tests +## Create testAuto script ### Usage -This function creates an MD file in the (user-selected) tests root folder. The file provides links to both the tests and the MD files. +This function creates a script named `testAuto.js` in a user-selected test folder. +### Details +The script created runs the `test.js` script in the folder in automatic mode. The script is the same for all tests. +## Create all testAuto scripts +### Usage +This function creates all testAuto scripts recursively from the user-selected root folder. This can be any folder in the tests hierarchy (e.g. all engine\material tests). + +The file provides a hierarchical list of all the tests ## Create Recursive Script ### Usage -After the user selects a folder within the tests hierarchy, a script is created, named `testRecursive.js`. This script calls all `test.js` scripts in the subfolders. +After the user selects a folder within the tests hierarchy, a script is created, named `testRecursive.js`. This script calls all `test.js` scripts in the sub-folders. ### Details The various scripts are called in alphabetical order. @@ -103,13 +113,13 @@ If any tests have failed, then a zipped folder will be created in the snapshots ### Usage Before starting the evaluation, make sure the GitHub user and branch are set correctly. The user should not normally be changed, but the branch may need to be set to the appropriate RC. -After setting the checkbox as required and pressing Evaluate - the user will be asked for the snapshots folder. +After setting the check-box as required and pressing Evaluate - the user will be asked for the snapshots folder. ### Details Evaluation proceeds in a number of steps: 1. A folder is created to store any failures -1. The expecetd images are download from GitHub. They are named slightly differently from the snapshots (e.g. `tests.engine.render.effect.highlight.coverage.00000.png` and `tests.engine.render.effect.highlight.coverage.00000_EI.png`). +1. The expected images are download from GitHub. They are named slightly differently from the snapshots (e.g. `tests.engine.render.effect.highlight.coverage.00000.png` and `tests.engine.render.effect.highlight.coverage.00000_EI.png`). 1. The images are then pair-wise compared, using the SSIM algorithm. A fixed threshold is used to define a mismatch. @@ -129,9 +139,9 @@ Any access to TestRail will require the TestRail account (default is High Fideli ![](./TestRailSelector.PNG) - The default test rail user is shown, and can be changed as needed. -- The username is usually the user's email. +- The user-name is usually the user's email. - The Project ID defaults to 14 - Interface. -- The Suite ID defaults to 1147 - Renderong. +- The Suite ID defaults to 1147 - Rendering. - The TestRail page provides 3 functions for writing to TestRail. ## Create Test Cases ### Usage diff --git a/tools/auto-tester/TestRail.PNG b/tools/auto-tester/TestRail.PNG index 042d0cf1cb..d7216e3c18 100644 Binary files a/tools/auto-tester/TestRail.PNG and b/tools/auto-tester/TestRail.PNG differ diff --git a/tools/auto-tester/Windows.PNG b/tools/auto-tester/Windows.PNG index bf7b76ba02..3faf5469bb 100644 Binary files a/tools/auto-tester/Windows.PNG and b/tools/auto-tester/Windows.PNG differ diff --git a/tools/auto-tester/src/Test.cpp b/tools/auto-tester/src/Test.cpp index 3da789f405..b7fe3a4f96 100644 --- a/tools/auto-tester/src/Test.cpp +++ b/tools/auto-tester/src/Test.cpp @@ -244,7 +244,7 @@ void Test::startTestsEvaluation(const QString& testFolder, const QString& branch void Test::finishTestsEvaluation(bool isRunningFromCommandline, bool interactiveMode, QProgressBar* progressBar) { bool success = compareImageLists((!isRunningFromCommandline && interactiveMode), progressBar); - if (interactiveMode && !isRunningFromCommandline) { + if (!isRunningFromCommandline) { if (success) { QMessageBox::information(0, "Success", "All images are as expected"); } else { @@ -302,174 +302,6 @@ void Test::includeTest(QTextStream& textStream, const QString& testPathname) { textStream << "Script.include(testsRootPath + \"" << partialPathWithoutTests + "\");" << endl; } -// Creates a single script in a user-selected folder. -// This script will run all text.js scripts in every applicable sub-folder -void Test::createRecursiveScript() { - // Select folder to start recursing from - QString previousSelection = _testDirectory; - QString parent = previousSelection.left(previousSelection.lastIndexOf('/')); - if (!parent.isNull() && parent.right(1) != "/") { - parent += "/"; - } - - _testDirectory = - QFileDialog::getExistingDirectory(nullptr, "Please select folder that will contain the top level test script", parent, - QFileDialog::ShowDirsOnly); - - // If user cancelled then restore previous selection and return - if (_testDirectory == "") { - _testDirectory = previousSelection; - return; - } - - createRecursiveScript(_testDirectory, true); -} - -// This method creates a `testRecursive.js` script in every sub-folder. -void Test::createAllRecursiveScripts() { - // Select folder to start recursing from - QString previousSelection = _testsRootDirectory; - QString parent = previousSelection.left(previousSelection.lastIndexOf('/')); - if (!parent.isNull() && parent.right(1) != "/") { - parent += "/"; - } - - _testsRootDirectory = QFileDialog::getExistingDirectory(nullptr, "Please select the root folder for the recursive scripts", - parent, QFileDialog::ShowDirsOnly); - - // If user cancelled then restore previous selection and return - if (_testsRootDirectory == "") { - _testsRootDirectory = previousSelection; - return; - } - - createRecursiveScript(_testsRootDirectory, false); - - QDirIterator it(_testsRootDirectory.toStdString().c_str(), QDirIterator::Subdirectories); - while (it.hasNext()) { - QString directory = it.next(); - - // Only process directories - QDir dir; - if (!isAValidDirectory(directory)) { - continue; - } - - // Only process directories that have sub-directories - bool hasNoSubDirectories{ true }; - QDirIterator it2(directory.toStdString().c_str(), QDirIterator::Subdirectories); - while (it2.hasNext()) { - QString directory2 = it2.next(); - - // Only process directories - QDir dir; - if (isAValidDirectory(directory2)) { - hasNoSubDirectories = false; - break; - } - } - - if (!hasNoSubDirectories) { - createRecursiveScript(directory, false); - } - } - - QMessageBox::information(0, "Success", "Scripts have been created"); -} - -void Test::createRecursiveScript(const QString& topLevelDirectory, bool interactiveMode) { - const QString recursiveTestsFilename("testRecursive.js"); - QFile allTestsFilename(topLevelDirectory + "/" + recursiveTestsFilename); - if (!allTestsFilename.open(QIODevice::WriteOnly | QIODevice::Text)) { - QMessageBox::critical(0, - "Internal error: " + QString(__FILE__) + ":" + QString::number(__LINE__), - "Failed to create \"" + recursiveTestsFilename + "\" in directory \"" + topLevelDirectory + "\"" - ); - - exit(-1); - } - - QTextStream textStream(&allTestsFilename); - - const QString DATE_TIME_FORMAT("MMM d yyyy, h:mm"); - textStream << "// This is an automatically generated file, created by auto-tester on " << QDateTime::currentDateTime().toString(DATE_TIME_FORMAT) << endl << endl; - - // Include 'autoTest.js' - QString branch = autoTester->getSelectedBranch(); - QString user = autoTester->getSelectedUser(); - - textStream << "PATH_TO_THE_REPO_PATH_UTILS_FILE = \"https://raw.githubusercontent.com/" + user + "/hifi_tests/" + branch + "/tests/utils/branchUtils.js\";" << endl; - textStream << "Script.include(PATH_TO_THE_REPO_PATH_UTILS_FILE);" << endl; - textStream << "var autoTester = createAutoTester(Script.resolvePath(\".\"));" << endl << endl; - - textStream << "var testsRootPath = autoTester.getTestsRootPath();" << endl << endl; - - // Wait 10 seconds before starting - textStream << "if (typeof Test !== 'undefined') {" << endl; - textStream << " Test.wait(10000);" << endl; - textStream << "};" << endl << endl; - - textStream << "autoTester.enableRecursive();" << endl; - textStream << "autoTester.enableAuto();" << endl << endl; - - // This is used to verify that the recursive test contains at least one test - bool testFound{ false }; - - // Directories are included in reverse order. The autoTester scripts use a stack mechanism, - // so this ensures that the tests run in alphabetical order (a convenience when debugging) - QStringList directories; - - // First test if top-level folder has a test.js file - const QString testPathname{ topLevelDirectory + "/" + TEST_FILENAME }; - QFileInfo fileInfo(testPathname); - if (fileInfo.exists()) { - // Current folder contains a test - directories.push_front(testPathname); - - testFound = true; - } - - QDirIterator it(topLevelDirectory.toStdString().c_str(), QDirIterator::Subdirectories); - while (it.hasNext()) { - QString directory = it.next(); - - // Only process directories - QDir dir(directory); - if (!isAValidDirectory(directory)) { - continue; - } - - const QString testPathname { directory + "/" + TEST_FILENAME }; - QFileInfo fileInfo(testPathname); - if (fileInfo.exists()) { - // Current folder contains a test - directories.push_front(testPathname); - - testFound = true; - } - } - - if (interactiveMode && !testFound) { - QMessageBox::information(0, "Failure", "No \"" + TEST_FILENAME + "\" files found"); - allTestsFilename.close(); - return; - } - - // Now include the test scripts - for (int i = 0; i < directories.length(); ++i) { - includeTest(textStream, directories.at(i)); - } - - textStream << endl; - textStream << "autoTester.runRecursive();" << endl; - - allTestsFilename.close(); - - if (interactiveMode) { - QMessageBox::information(0, "Success", "Script has been created"); - } -} - void Test::createTests() { // Rename files sequentially, as ExpectedResult_00000.jpeg, ExpectedResult_00001.jpg and so on // Any existing expected result images will be deleted @@ -613,9 +445,7 @@ ExtractedText Test::getTestScriptLines(QString testFileName) { return relevantTextFromTest; } -// Create an MD file for a user-selected test. -// The folder selected must contain a script named "test.js", the file produced is named "test.md" -void Test::createMDFile() { +bool Test::createFileSetup() { // Folder selection QString previousSelection = _testDirectory; QString parent = previousSelection.left(previousSelection.lastIndexOf('/')); @@ -624,20 +454,18 @@ void Test::createMDFile() { } _testDirectory = QFileDialog::getExistingDirectory(nullptr, "Please select folder containing the test", parent, - QFileDialog::ShowDirsOnly); + QFileDialog::ShowDirsOnly); // If user cancelled then restore previous selection and return if (_testDirectory == "") { _testDirectory = previousSelection; - return; + return false; } - createMDFile(_testDirectory); - - QMessageBox::information(0, "Success", "MD file has been created"); + return true; } -void Test::createAllMDFiles() { +bool Test::createAllFilesSetup() { // Select folder to start recursing from QString previousSelection = _testsRootDirectory; QString parent = previousSelection.left(previousSelection.lastIndexOf('/')); @@ -646,11 +474,31 @@ void Test::createAllMDFiles() { } _testsRootDirectory = QFileDialog::getExistingDirectory(nullptr, "Please select the root folder for the MD files", parent, - QFileDialog::ShowDirsOnly); + QFileDialog::ShowDirsOnly); // If user cancelled then restore previous selection and return if (_testsRootDirectory == "") { _testsRootDirectory = previousSelection; + return false; + } + + return true; +} + +// Create an MD file for a user-selected test. +// The folder selected must contain a script named "test.js", the file produced is named "test.md" +void Test::createMDFile() { + if (!createFileSetup()) { + return; + } + + if (createMDFile(_testDirectory)) { + QMessageBox::information(0, "Success", "MD file has been created"); + } +} + +void Test::createAllMDFiles() { + if (!createAllFilesSetup()) { return; } @@ -681,18 +529,18 @@ void Test::createAllMDFiles() { QMessageBox::information(0, "Success", "MD files have been created"); } -void Test::createMDFile(const QString& _testDirectory) { +bool Test::createMDFile(const QString& directory) { // Verify folder contains test.js file - QString testFileName(_testDirectory + "/" + TEST_FILENAME); + QString testFileName(directory + "/" + TEST_FILENAME); QFileInfo testFileInfo(testFileName); if (!testFileInfo.exists()) { QMessageBox::critical(0, "Error", "Could not find file: " + TEST_FILENAME); - return; + return false; } ExtractedText testScriptLines = getTestScriptLines(testFileName); - QString mdFilename(_testDirectory + "/" + "test.md"); + QString mdFilename(directory + "/" + "test.md"); QFile mdFile(mdFilename); if (!mdFile.open(QIODevice::WriteOnly)) { QMessageBox::critical(0, "Internal error: " + QString(__FILE__) + ":" + QString::number(__LINE__), "Failed to create file " + mdFilename); @@ -727,6 +575,217 @@ void Test::createMDFile(const QString& _testDirectory) { } mdFile.close(); + return true; +} + +void Test::createTestAutoScript() { + if (!createFileSetup()) { + return; + } + + if (createTestAutoScript(_testDirectory)) { + QMessageBox::information(0, "Success", "'autoTester.js` script has been created"); + } +} + +void Test::createAllTestAutoScripts() { + if (!createAllFilesSetup()) { + return; + } + + // First test if top-level folder has a test.js file + const QString testPathname{ _testsRootDirectory + "/" + TEST_FILENAME }; + QFileInfo fileInfo(testPathname); + if (fileInfo.exists()) { + createTestAutoScript(_testsRootDirectory); + } + + QDirIterator it(_testsRootDirectory.toStdString().c_str(), QDirIterator::Subdirectories); + while (it.hasNext()) { + QString directory = it.next(); + + // Only process directories + QDir dir; + if (!isAValidDirectory(directory)) { + continue; + } + + const QString testPathname{ directory + "/" + TEST_FILENAME }; + QFileInfo fileInfo(testPathname); + if (fileInfo.exists()) { + createTestAutoScript(directory); + } + } + + QMessageBox::information(0, "Success", "'autoTester.js' scripts have been created"); +} + +bool Test::createTestAutoScript(const QString& directory) { + // Verify folder contains test.js file + QString testFileName(directory + "/" + TEST_FILENAME); + QFileInfo testFileInfo(testFileName); + if (!testFileInfo.exists()) { + QMessageBox::critical(0, "Error", "Could not find file: " + TEST_FILENAME); + return false; + } + + QString testAutoScriptFilename(directory + "/" + "testAuto.js"); + QFile testAutoScriptFile(testAutoScriptFilename); + if (!testAutoScriptFile.open(QIODevice::WriteOnly)) { + QMessageBox::critical(0, "Internal error: " + QString(__FILE__) + ":" + QString::number(__LINE__), + "Failed to create file " + testAutoScriptFilename); + exit(-1); + } + + QTextStream stream(&testAutoScriptFile); + + stream << "if (typeof PATH_TO_THE_REPO_PATH_UTILS_FILE === 'undefined') PATH_TO_THE_REPO_PATH_UTILS_FILE = 'https://raw.githubusercontent.com/highfidelity/hifi_tests/master/tests/utils/branchUtils.js';\n"; + stream << "Script.include(PATH_TO_THE_REPO_PATH_UTILS_FILE);\n"; + stream << "var autoTester = createAutoTester(Script.resolvePath('.'));\n\n"; + stream << "autoTester.enableAuto();\n\n"; + stream << "Script.include('./test.js?raw=true');\n"; + + testAutoScriptFile.close(); + return true; +} + +// Creates a single script in a user-selected folder. +// This script will run all text.js scripts in every applicable sub-folder +void Test::createRecursiveScript() { + if (!createFileSetup()) { + return; + } + + createRecursiveScript(_testDirectory, true); + QMessageBox::information(0, "Success", "'testRecursive.js` script has been created"); +} + +// This method creates a `testRecursive.js` script in every sub-folder. +void Test::createAllRecursiveScripts() { + if (!createAllFilesSetup()) { + return; + } + + createRecursiveScript(_testsRootDirectory, false); + + QDirIterator it(_testsRootDirectory.toStdString().c_str(), QDirIterator::Subdirectories); + while (it.hasNext()) { + QString directory = it.next(); + + // Only process directories + QDir dir; + if (!isAValidDirectory(directory)) { + continue; + } + + // Only process directories that have sub-directories + bool hasNoSubDirectories{ true }; + QDirIterator it2(directory.toStdString().c_str(), QDirIterator::Subdirectories); + while (it2.hasNext()) { + QString directory2 = it2.next(); + + // Only process directories + QDir dir; + if (isAValidDirectory(directory2)) { + hasNoSubDirectories = false; + break; + } + } + + if (!hasNoSubDirectories) { + createRecursiveScript(directory, false); + } + } + + QMessageBox::information(0, "Success", "Scripts have been created"); +} + +void Test::createRecursiveScript(const QString& topLevelDirectory, bool interactiveMode) { + const QString recursiveTestsFilename("testRecursive.js"); + QFile allTestsFilename(topLevelDirectory + "/" + recursiveTestsFilename); + if (!allTestsFilename.open(QIODevice::WriteOnly | QIODevice::Text)) { + QMessageBox::critical(0, "Internal error: " + QString(__FILE__) + ":" + QString::number(__LINE__), + "Failed to create \"" + recursiveTestsFilename + "\" in directory \"" + topLevelDirectory + "\""); + + exit(-1); + } + + QTextStream textStream(&allTestsFilename); + + textStream << "// This is an automatically generated file, created by auto-tester" << endl; + + // Include 'autoTest.js' + QString branch = autoTester->getSelectedBranch(); + QString user = autoTester->getSelectedUser(); + + textStream << "PATH_TO_THE_REPO_PATH_UTILS_FILE = \"https://raw.githubusercontent.com/" + user + "/hifi_tests/" + branch + + "/tests/utils/branchUtils.js\";" + << endl; + textStream << "Script.include(PATH_TO_THE_REPO_PATH_UTILS_FILE);" << endl; + textStream << "var autoTester = createAutoTester(Script.resolvePath(\".\"));" << endl << endl; + + textStream << "var testsRootPath = autoTester.getTestsRootPath();" << endl << endl; + + // Wait 10 seconds before starting + textStream << "if (typeof Test !== 'undefined') {" << endl; + textStream << " Test.wait(10000);" << endl; + textStream << "};" << endl << endl; + + textStream << "autoTester.enableRecursive();" << endl; + textStream << "autoTester.enableAuto();" << endl << endl; + + // This is used to verify that the recursive test contains at least one test + bool testFound{ false }; + + // Directories are included in reverse order. The autoTester scripts use a stack mechanism, + // so this ensures that the tests run in alphabetical order (a convenience when debugging) + QStringList directories; + + // First test if top-level folder has a test.js file + const QString testPathname{ topLevelDirectory + "/" + TEST_FILENAME }; + QFileInfo fileInfo(testPathname); + if (fileInfo.exists()) { + // Current folder contains a test + directories.push_front(testPathname); + + testFound = true; + } + + QDirIterator it(topLevelDirectory.toStdString().c_str(), QDirIterator::Subdirectories); + while (it.hasNext()) { + QString directory = it.next(); + + // Only process directories + QDir dir(directory); + if (!isAValidDirectory(directory)) { + continue; + } + + const QString testPathname{ directory + "/" + TEST_FILENAME }; + QFileInfo fileInfo(testPathname); + if (fileInfo.exists()) { + // Current folder contains a test + directories.push_front(testPathname); + + testFound = true; + } + } + + if (interactiveMode && !testFound) { + QMessageBox::information(0, "Failure", "No \"" + TEST_FILENAME + "\" files found"); + allTestsFilename.close(); + return; + } + + // Now include the test scripts + for (int i = 0; i < directories.length(); ++i) { + includeTest(textStream, directories.at(i)); + } + + textStream << endl; + textStream << "autoTester.runRecursive();" << endl; + + allTestsFilename.close(); } void Test::createTestsOutline() { diff --git a/tools/auto-tester/src/Test.h b/tools/auto-tester/src/Test.h index 853e9c98e2..1984ede234 100644 --- a/tools/auto-tester/src/Test.h +++ b/tools/auto-tester/src/Test.h @@ -46,22 +46,29 @@ public: void startTestsEvaluation(const QString& testFolder = QString(), const QString& branchFromCommandLine = QString(), const QString& userFromCommandLine = QString()); void finishTestsEvaluation(bool isRunningFromCommandline, bool interactiveMode, QProgressBar* progressBar); - void createRecursiveScript(); - void createAllRecursiveScripts(); - void createRecursiveScript(const QString& topLevelDirectory, bool interactiveMode); - void createTests(); void createTestsOutline(); + bool createFileSetup(); + bool createAllFilesSetup(); + void createMDFile(); void createAllMDFiles(); - void createMDFile(const QString& topLevelDirectory); + bool createMDFile(const QString& directory); + + void createTestAutoScript(); + void createAllTestAutoScripts(); + bool createTestAutoScript(const QString& directory); void createTestRailTestCases(); void createTestRailRun(); void updateTestRailRunResult(); + void createRecursiveScript(); + void createAllRecursiveScripts(); + void createRecursiveScript(const QString& topLevelDirectory, bool interactiveMode); + bool compareImageLists(bool isInteractiveMode, QProgressBar* progressBar); QStringList createListOfAll_imagesInDirectory(const QString& imageFormat, const QString& pathToImageDirectory); diff --git a/tools/auto-tester/src/TestRailInterface.cpp b/tools/auto-tester/src/TestRailInterface.cpp index 93559490e5..5c950e0817 100644 --- a/tools/auto-tester/src/TestRailInterface.cpp +++ b/tools/auto-tester/src/TestRailInterface.cpp @@ -25,7 +25,7 @@ TestRailInterface::TestRailInterface() { _testRailTestCasesSelectorWindow.setUser("@highfidelity.io"); ////_testRailTestCasesSelectorWindow.setUser("nissim.hadar@gmail.com"); - _testRailTestCasesSelectorWindow.setProjectID(INTERFACE_PROJECT_ID); + _testRailTestCasesSelectorWindow.setProjectID(INTERFACE_AUTOMATION_PROJECT_ID); ////_testRailTestCasesSelectorWindow.setProjectID(2); _testRailTestCasesSelectorWindow.setSuiteID(INTERFACE_SUITE_ID); @@ -36,7 +36,7 @@ TestRailInterface::TestRailInterface() { _testRailRunSelectorWindow.setUser("@highfidelity.io"); ////_testRailRunSelectorWindow.setUser("nissim.hadar@gmail.com"); - _testRailRunSelectorWindow.setProjectID(INTERFACE_PROJECT_ID); + _testRailRunSelectorWindow.setProjectID(INTERFACE_AUTOMATION_PROJECT_ID); ////_testRailRunSelectorWindow.setProjectID(2); _testRailRunSelectorWindow.setSuiteID(INTERFACE_SUITE_ID); @@ -47,7 +47,7 @@ TestRailInterface::TestRailInterface() { _testRailResultsSelectorWindow.setUser("@highfidelity.io"); ////_testRailResultsSelectorWindow.setUser("nissim.hadar@gmail.com"); - _testRailResultsSelectorWindow.setProjectID(INTERFACE_PROJECT_ID); + _testRailResultsSelectorWindow.setProjectID(INTERFACE_AUTOMATION_PROJECT_ID); ////_testRailResultsSelectorWindow.setProjectID(2); _testRailResultsSelectorWindow.setSuiteID(INTERFACE_SUITE_ID); @@ -908,7 +908,7 @@ QDomElement TestRailInterface::processTestXML(const QString& fullDirectory, ++i; QString title{ words[i] }; for (++i; i < words.length() - 1; ++i) { - title += " / " + words[i]; + title += "/" + words[i]; } QDomElement titleElement = _document.createElement("title"); diff --git a/tools/auto-tester/src/TestRailInterface.h b/tools/auto-tester/src/TestRailInterface.h index 6f250dfbba..ba5e94957b 100644 --- a/tools/auto-tester/src/TestRailInterface.h +++ b/tools/auto-tester/src/TestRailInterface.h @@ -92,10 +92,10 @@ public: private: // HighFidelity Interface project ID in TestRail - const int INTERFACE_PROJECT_ID{ 24 }; + const int INTERFACE_AUTOMATION_PROJECT_ID{ 26 }; // Rendering suite ID - const int INTERFACE_SUITE_ID{ 1147 }; + const int INTERFACE_SUITE_ID{ 1312 }; QDomDocument _document; diff --git a/tools/auto-tester/src/ui/AutoTester.cpp b/tools/auto-tester/src/ui/AutoTester.cpp index 13bda4853f..9e8aa406b8 100644 --- a/tools/auto-tester/src/ui/AutoTester.cpp +++ b/tools/auto-tester/src/ui/AutoTester.cpp @@ -78,6 +78,14 @@ void AutoTester::on_createAllMDFilesButton_clicked() { _test->createAllMDFiles(); } +void AutoTester::on_createTestAutoScriptButton_clicked() { + _test->createTestAutoScript(); +} + +void AutoTester::on_createAllTestAutoScriptsButton_clicked() { + _test->createAllTestAutoScripts(); +} + void AutoTester::on_createTestsOutlineButton_clicked() { _test->createTestsOutline(); } diff --git a/tools/auto-tester/src/ui/AutoTester.h b/tools/auto-tester/src/ui/AutoTester.h index e29da5b716..26eec6f07f 100644 --- a/tools/auto-tester/src/ui/AutoTester.h +++ b/tools/auto-tester/src/ui/AutoTester.h @@ -50,6 +50,9 @@ private slots: void on_createMDFileButton_clicked(); void on_createAllMDFilesButton_clicked(); + void on_createTestAutoScriptButton_clicked(); + void on_createAllTestAutoScriptsButton_clicked(); + void on_createTestsOutlineButton_clicked(); void on_createTestRailTestCasesButton_clicked(); diff --git a/tools/auto-tester/src/ui/AutoTester.ui b/tools/auto-tester/src/ui/AutoTester.ui index ac8fcf5e86..576ad14aae 100644 --- a/tools/auto-tester/src/ui/AutoTester.ui +++ b/tools/auto-tester/src/ui/AutoTester.ui @@ -6,7 +6,7 @@ 0 0 - 432 + 582 734 @@ -23,8 +23,8 @@ - 166 - 610 + 235 + 620 100 40 @@ -36,9 +36,9 @@ - 12 + 30 140 - 408 + 521 461 @@ -52,7 +52,7 @@ - 96 + 145 20 220 40 @@ -65,8 +65,8 @@ - 96 - 100 + 20 + 140 220 40 @@ -78,8 +78,8 @@ - 96 - 150 + 270 + 140 220 40 @@ -91,8 +91,8 @@ - 96 - 230 + 145 + 80 220 40 @@ -104,8 +104,8 @@ - 96 - 310 + 20 + 260 220 40 @@ -117,8 +117,8 @@ - 96 - 360 + 270 + 260 220 40 @@ -127,6 +127,32 @@ Create all Recursive Scripts + + + + 20 + 200 + 220 + 40 + + + + Create testAuto script + + + + + + 270 + 200 + 220 + 40 + + + + Create all testAuto scripts + + @@ -135,8 +161,8 @@ - 90 - 100 + 150 + 230 255 23 @@ -148,8 +174,8 @@ - 90 - 50 + 150 + 180 131 20 @@ -164,8 +190,8 @@ - 200 - 40 + 260 + 170 101 40 @@ -182,8 +208,8 @@ - 180 - 160 + 210 + 230 161 40 @@ -195,8 +221,8 @@ - 80 - 40 + 110 + 110 95 20 @@ -211,8 +237,8 @@ - 180 - 100 + 210 + 170 161 40 @@ -224,8 +250,8 @@ - 180 - 40 + 210 + 110 161 40 @@ -237,8 +263,8 @@ - 80 - 60 + 110 + 130 95 20 @@ -255,8 +281,8 @@ - 100 - 100 + 160 + 130 211 40 @@ -268,8 +294,8 @@ - 100 - 170 + 160 + 200 211 40 @@ -283,8 +309,8 @@ - 110 - 90 + 160 + 80 81 16 @@ -301,8 +327,8 @@ - 200 - 85 + 250 + 75 140 24 @@ -311,8 +337,8 @@ - 200 - 47 + 250 + 37 140 24 @@ -321,8 +347,8 @@ - 110 - 50 + 160 + 40 81 16 @@ -342,7 +368,7 @@ 0 0 - 432 + 582 21