Kasutajaliidesepõhiste automatiseeritud regressioontestide komplekti loomine
Files
Date
2013
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Tartu Ülikool
Abstract
Antud töö esimeses osas tutvustati regressioontestimist üldisemalt, selgitades selle vajadust tulenevalt pidevalt muutuvast lähtekoodist ja sellega kaasnevast vigade ohust. Analüüsiti, kuidas testjuhtumite loomine aitab kaasa regressioontestimisele, sest nende loomisel teostatakse planeerimise osa funktsionaalsuste testimiseks ning tulevikus saab loodud teste kergesti taaskasutada. Toodi välja kaudseid spetsifikatsioone nagu konkureerivad tooted, kasutajate kommentaarid, kasutajaliidese standardid ja testijate enda kogemus, mida võiks testjuhtumite kirjutamisel kasutada, ning selgitati, kuidas kirjutatava dokumendi detailsus peaks sõltuma sellest, kas loodav testjuhtum on manuaalseks testimiseks mõeldud või kasutatav sisendina automatiseerimisele. Eraldi analüüsiti väljakutseid, mida regressioontestimine endaga kaasa toob ning erinevaid tehnikaid regressioontestimise läbiviimiseks. Üks suurimaid probleeme oli risk, et tähtis osa funktsionaalsusest jääb testimata tulenevalt ajapuudusest. Selle riski vähendamiseks tutvustati lähenemist, kus testjuhtumitele määratakse prioriteedid. Tulemusena on testijal alati teada, millised funktsionaalsused vajavad enim tähelepanu, ning nende testimisele pannakse suuremat rõhku kui nende funktsionaalsuste testimisele, mis omavad madalamat prioriteeti.
Teises osas tutvustati testide automatiseerimist, mis aitab tegeleda kasvava funktsionaalsuse, rutiinseks kujuneva töö ja võimalike inimvigadega testimisel. Selgitati, et kuigi automatiseerimine on võimas tööriist, siis valesti kasutamisel võib see projektile rohkem kahju kui kasu tuua, raisates ressursse, mida saaks mujal paremini kasutada. Vältida tuleks selliseid eemärke nagu kogu rakenduse testimise automatiseerimine, kuna enamikel juhtudel eksisteerib funktsionaalsuseid, kus manuaalne testimine annab paremaid tulemusi kui automatiseeritud testimine. Tähtis on meeles pidada, et automatiseerimine ei õpeta testijaid paremini testima, seega tuleks enne automatiseerimist üle vaadata projekti käesolev testimisprotsess ning olemasolevate probleemide korral need enne ära lahendada. Valides testjuhtumeid, mida automatiseerida, tuleks mõelda, kas antud funktsionaalsuse testimine vajab inimlikku lähenemist ja võimet erinevaid otsuseid teha tulenevalt saadavatest tulemustest. Kui jah, siis tuleks eelistada manuaalset testimist, sest automaattestid teevad ainult seda, mis neile ette on öeldud. Samuti tuleks mõelda, kas saadava tulemuse õigsust on arvutiprogrammil lihtne hinnata. Näiteks piltide puhul on tulemust arvutil väga keeruline hinnata ning selliseid teste
36
peaks läbi viima inimene. Testjuhtumid, mis omavad suuri andmemahte ja mille läbiviimine on väga rutiine töö, sobivad hästi automatiseerimiseks, eeldusel, et funktsionaalsus on stabiilne ning seda ei muudeta lähitulevikus. Testjuhtumite koostamine ning hooldamine peaks jääma testija vastutuseks ning automaattestide loomine ja täiendamine võiks olla arendaja töö, olenevalt sellest, milliste oskustega inimesed projektis on.
Kolmandas osas tutvustati Selenium Webdriver tööriista, mida kasutati antud töö raames loodud testide automatiseerimiseks. Antud tööriist valiti, kuna ta võimaldab automatiseerida funktsionaalseid teste, kirjutada teste programmeerimiskeeles Java, genereerida tulemustest raporteid ning on vabavaraline tarkvara. Lühidalt tutvustati andmetepõhist testimist ning tähtsamatest Webdriveri funktsionaalsustest uuriti automaattestide ootamise loogika loomise, veebielementide asukoha määramise ja tulemuste kontrollimise võimalusi. Lisaks kirjeldati eraldi Selenium IDE tööriista, mida saab kasutada algsete testide automatiseerimiseks või abivahendina keerulisemate testide loomisel. Loodi reaalne juhis Webdriver testide käitamiseks Eclipse arenduskeskkonnas ning kirjeldati, millised sammud tuleb läbida, kui eksisteerivad testid, mille käitamiseks kasutatakse Apache Ant tarkvara (töö käigus ei kirjeldatud, kuidas antud tarkvara seadistada testidega suhtlemiseks). Lühidalt tutvustati ka TestNG ning ReportNG tarkvara, mida saab kasutada testide omavaheliste sõltuvuste määramiseks ning loetavamate raportite koostamiseks.
Töö tulemusena valmis algne kasutajaliidesepõhiste automaattestide komplekt, mida saab EAK projektis kasutada, abistamaks regressioontestimist ning kiirendamaks arendusprotsessi, andes tekkivate vigade korral sellest võimalikult vara teada. Automaattestide komplekt sisaldab teste sisselogimise, müügiarvete sisestamise ja ostuarvete sisestamise funktsionaalsuste kohta. Samuti valmisid testjuhtumid ja nimekiri rakenduse funktsionaalsustest, mida saab kasutada regressioontestimise plaanina.
The main goal of this Thesis was to create initial user interface based automated regression test suite for EAK project to cope with testing of growing functionality and speed up the developement process. First, the regression testing was examined to understand that it is needed because possibility to introduce new bugs to old functionality while developing new components for application. While regression testing is vital part of software testing there are also some problems that it will create. One major problem was risk that critical bugs would not be found because there would not be enough time to test everything. To deal with this risk, the idea of priority assigning to tests cases was introduced. Having priorities, the tester will test most important functionality first and if time runs out then functionalities with lower priorities would not be tested as thoroughly as were test cases with higher priorities. At second part of this Thesis automation was introduced to deal with growing functionality, routine work and human errors at testing. While automation is powerful tool to use if it is used in the correct way it also may cause loss of valuble resources if it is done in the wrong way. Most important thing is to remember that automation will not teach testers how to test, so testing process should be reviewed and if there are any existing problems then they should be dealt before starting to automate tests. In most cases it is not realistic to automate all the testing in project, so tests for automation should be chosen carefully. Testers should think if testing of some functionality requires exploratory testing, if yes then it is not a good candidate for automation. Also if output of test is hard to confirm by automated test (for example picture is generated) then it should be tested manually. But test cases that have large amount of data or that require a lot of routine are excelent candidates for automation (assuming that components to be tested are very stable and will not be changed in near future). 38 At third part of this Thesis Selenium Webdriver was introduced as the automation tool which was used to create automated tests for EAK project. It was explained that this tool was selected because of its ability to automate functional tests, posibility to create tests in Java programming language, report generating ability and because it is freeware application. Overview of Selenium IDE was given, that can be used to record trivial automated tests or to use as supportive tool for creating more complex tests with Webdriver. Data driven testing was shortly intorduced and also most important parts of Webdriver functionality were investigated like what possibilities are to create waiting logic, how to locate webelements and how to confirm results. It was explained how to set up environment for developing Webdriver tests in Eclipse developing environment. TestNG and ReportNG frameworks were intorduced to create dependencies between automated tests and to generate more readable reports. As result of this thesis initial set of automated regression tests were created for EAK project along with test cases and list of application functionalities that can be used as regression test plan.
The main goal of this Thesis was to create initial user interface based automated regression test suite for EAK project to cope with testing of growing functionality and speed up the developement process. First, the regression testing was examined to understand that it is needed because possibility to introduce new bugs to old functionality while developing new components for application. While regression testing is vital part of software testing there are also some problems that it will create. One major problem was risk that critical bugs would not be found because there would not be enough time to test everything. To deal with this risk, the idea of priority assigning to tests cases was introduced. Having priorities, the tester will test most important functionality first and if time runs out then functionalities with lower priorities would not be tested as thoroughly as were test cases with higher priorities. At second part of this Thesis automation was introduced to deal with growing functionality, routine work and human errors at testing. While automation is powerful tool to use if it is used in the correct way it also may cause loss of valuble resources if it is done in the wrong way. Most important thing is to remember that automation will not teach testers how to test, so testing process should be reviewed and if there are any existing problems then they should be dealt before starting to automate tests. In most cases it is not realistic to automate all the testing in project, so tests for automation should be chosen carefully. Testers should think if testing of some functionality requires exploratory testing, if yes then it is not a good candidate for automation. Also if output of test is hard to confirm by automated test (for example picture is generated) then it should be tested manually. But test cases that have large amount of data or that require a lot of routine are excelent candidates for automation (assuming that components to be tested are very stable and will not be changed in near future). 38 At third part of this Thesis Selenium Webdriver was introduced as the automation tool which was used to create automated tests for EAK project. It was explained that this tool was selected because of its ability to automate functional tests, posibility to create tests in Java programming language, report generating ability and because it is freeware application. Overview of Selenium IDE was given, that can be used to record trivial automated tests or to use as supportive tool for creating more complex tests with Webdriver. Data driven testing was shortly intorduced and also most important parts of Webdriver functionality were investigated like what possibilities are to create waiting logic, how to locate webelements and how to confirm results. It was explained how to set up environment for developing Webdriver tests in Eclipse developing environment. TestNG and ReportNG frameworks were intorduced to create dependencies between automated tests and to generate more readable reports. As result of this thesis initial set of automated regression tests were created for EAK project along with test cases and list of application functionalities that can be used as regression test plan.