The approach used in this solution is to create an automatic system to retrieve and execute tests inside testsets. The idea behind is to call something that do the "job", without need to intervene in the middle of the process.
The Solution
When we start to work on this task we expose some requirements:
Phases
There are 4 macro phases in this solutions that could be summarize like in this picture
REST API
The first step consists on a series of REST calls to retrieve data from ALM. This procedure must be called, if we suppose the same structure of the previous article, with these parameters:
This procedure must retrieve the entire path of the TestSets, in a json format, as {"TestSets":["Root\\<APPNAME>\\<ENVIRONMENT>\\TestSetName1","Root\\<APPNAME>\\<ENVIRONMENT>\\TestSetName2",..., "Root\\<APPNAME>\\<ENVIRONMENT>\\TestSetNameN"]} and pass this information to the other steps that are in a single jenkins pipeline.
the REST API sequence for this step must be:
1. The REST Authentication
Every time we have to interact with ALM we have 1st to authenticate. This operation can be done with a POST call to send user and password in an XML format. This can be done with this vbscript code:
Const RC_OK = 0
Const RC_KO = 3
Dim errCode : errCode = 3
'Retrieve data from the arguments passed to the script
if wscript.arguments.count <> 7 then
wscript.quit errCode
end if
'if we are here means that the numbers of parameters are OK
Dim ALMSRV : ALMSRV = wscript.arguments(0)
Dim ALMDOM : ALMDOM = wscript.arguments(1)
Dim ALMPRJ : ALMPRJ = wscript.arguments(2)
Dim ALMUSR: ALMUSR = wscript.arguments(3)
Dim ALMPWD : ALMPWD = wscript.arguments(4)
Dim APPNAME : APPNAME = wscript.arguments(5)
Dim APPENV : APPENV = wscript.arguments(6)
Const AUTH_CALL = "/authentication-point/alm-authenticate"
Const SESSION = "/rest/site-session"
'Define of the object to do http requests
Dim objSrvHTTP
set objSrvHTTP = CreateObject("Msxml2.ServerXMLHTTP.6.0")
Dim bolAuth
bolAuth=ExecAuth("POST", ALMSRV & AUTH_CALL, false, ALMUSR, ALMPWD)
if not(bolAuth) then
msgbox "Authentication Error - exit program"
set objSrvHTTP = Nothing
wscript.quit errCode
end if
'Retrieve the Session Cookie
Dim theSession : theSession = initSession("POST", ALMSRV & SESSION, false)
if theSession = "" then
msgbox "Session Creation Error - exit program"
set objSrvHTTP = Nothing
wscript.quit errCode
end if
function ExecAuth(method, strURL, bolAsyncCall, user, pass)
dim bolRes : bolRes = false
dim DataToSend :
DataToSend = "<alm-authentication> " & _
"<user>" & user & "</user>" & _
"<password>" & pass & "</password>" & _
"</alm-authentication>"
objSrvHTTP.open method, strURL, bolAsyncCall
objSrvHTTP.setRequestHeader "Content-Type", "application/xml"
objSrvHTTP.send DataToSend
if objSrvHTTP.status = 200 then
bolRes = true
end if
ExecAuth = bolRes
end function
function initSession(method, strURL, bolAsyncCall)
Const CLIENT_TYPE = "REST Client"
Const TIME_OUT = 10
Const QCSESSION_CK = "QCSession"
Dim res : res = ""
Dim DataToSend
DataToSend = "<session-parameters> " & _
"<client-type>" & CLIENT_TYPE & "</client-type>" & _
"<time-out>" & TIME_OUT & "</time-out>" & _
"</session-parameters>"
Dim strHDRS
Dim arrHdrs
objSrvHTTP.open metodo, indirizzo, chiamataAsincrona
objSrvHTTP.Send DataToSend
if objSrvHTTP.status = 201 then
strHDRS = objSrvHTTP.getAllResponseHeaders
arrHdrs = split(strHDRS, vbCrLF)
for i=0 to Ubound(arrHdrs)
if instr(arrHdrs(i),"Set-Cookie") > 0 then
if instr(arrHdrs(i), QCSESSION_CK) then
res = right(split(arrHdrs(i), ":")(1), len(split(arrHdrs(i), ":")(1)) - (len(QCSESSION_CK) + 2)) '2 because of '= '
end if
end if
next
else
msgbox "Error"
end if
initSession = res
end function
2. Retrieve TestSet(s) Info
As we know, the structure is known because in our example is "Root" + APPNAME + APPENV so we'll declare a variable that store this information and retrieve testsets and all their paths. The code is similar to this:
Dim TL_PATH : TL_PATH = Replace("Root\" & APPNAME & "\" & APPENV," ","*")
'the replace is needed for the REST query in case there are spaces in the path.
Dim AllTheTestSetPath : AllTheTestSetPath = "{" & chr(34) "TestSets" & chr(34) &
":["
From the TL_PATH I have to retrieve all the testsets. To do so I need to know the ID of the last TestLab folder, in our case is APPENV. With this information I'll be able to do a REST Query and know which are the testset under it.
Dim intIDFolderTL : intIDFolderTL = RetrieveIDLastFolder(TL_PATH)
Dim strTestSetNames : strTestSetNames = RetrieveTSNameByTLFolder(intIDFolderTL)
Dim arrTSN
if strTestSetNames <> "" then
if instr(strTestSetNames, ";") > 0 then
arrTSN = split(strTestSetNames, ";")
for u=0 to ubound(arrTSN)
AllTheTestSetPath = AllTheTestSetPath & chr(34) & Replace(TL_PATH,"*"," ") & "\" & arrTSN(u) & chr(34) & ","
next
AllTheTestSetPath = left(AllTheTestSetPath, len(AllTheTestSetPath) -1)
AllTheTestSetPath = AllTheTestSetPath & "]}"
else
AllTheTestSetPath = AllTheTestSetPath & chr(34) & Replace(TL_PATH,"*"," ") & "\" & strTestSetNames & chr(34) & "]}"
end if
end if
Dim myShell : set myShell = CreateObject("WScript.Shell")
if AllTheTestSetPath <> "" then
myShell.popup "TestSets retrieved: " & vbNewLine & vbNewLine & AllTheTestSetPath, 40,"TestSetPath",64
else
myShell.popup "No TestSets retrieved!!! Error!!!", 40,"TestSetPath",16
end if
'LOGOUT
ExecLogout "GET", QC_ADDRESS & LOGOUT_CALL, false
set objSrvHTTP = nothing
set myShell = nothing
wscript.quit RC_OK
'***************************************************
' FUNCTION SECTION
'***************************************************
Sub ExecLogout(method, q, bolAsync)
objSrvHTTP.open method, q, bolAsync
objSrvHTTP.send
if objSrvHTTP.status = 200 then
myShell.popup "Logout, 5,"Logout",64
end if
End Sub
Function NodeListByName(oRoot, nodeName)
Dim oNodeList
set oNodeList = oRoot.selectNodes("//" & nodeName)
set NodeListByName= oNodeList
set oNodeList = Nothing
End Function
Function RetrieveIDLastFolder(FolderPath)
Dim intIDFold : intIDFold = 0
Dim arrPath
if instr(FolderPath, "\") > 0 then
arrPath = split(FolderPath, "\")
for i=1 to Ubound(arrPath)
intIDFold = RecuperaIDFold(arrPath(i), intIDFold)
next
end if
RetrieveIDLastFolder = intIDFold
End Function
Function RecuperaIDFold(strFoldName, idFather)
Dim intRes : intRes = -99
Dim tempQRY
Dim QR_FOLD_ID : QR_FOLD_ID = "/test-set-folders?fields=id&query={parent-id[" & idFather & "];name[" & strFoldName & "]}"
tempQRY = QC_ADDRESS & restDomPrj & QR_FOLD_ID
objSrvHTTP.Open "GET", tempQRY, false
objSrvHTTP.Send
if objSrvHTTP.status = 200 then
set objXMLDoc=CreateObject("Msxml2.DOMDocument")
objXMLDoc.async = false
objXMLDoc.loadXML(objSrvHTTP.responseText)
set myRoot = objXMLDoc.documentElement
set theFieldNodeList = NodeListByName(myRoot, "Field")
if theFieldNodeList.length > 0 then
for each n in theFieldNodeList
for each attr in n.attributes
if attr.Value = "id" then
intRes = CInt(n.text)
end if
next
next
end if
set theFieldNodeList = Nothing
set myRoot = Nothing
end if
RecuperaIDFold = intRes
End Function
Function RetrieveTSNameByTLFolder(idF)
Dim strTSNames : strTSNames = ""
Dim QR_TSET_BYTLID : QR_TSET_BYTLID = "/test-sets?fields=name&query={parent-id[" & idF & "]}"
Dim tempQRY
tempQRY = QC_ADDRESS & restDomPrj & QR_TSET_BYTLID
objSrvHTTP.Open "GET", tempQRY, false
objSrvHTTP.Send
if objSrvHTTP.status = 200 then
set objXMLDoc=CreateObject("Msxml2.DOMDocument")
objXMLDoc.async = false
objXMLDoc.loadXML(objSrvHTTP.responseText)
set myRoot = objXMLDoc.documentElement
set theFieldNodeList = NodeListByName(myRoot, "Field")
if theFieldNodeList.length > 0 then
for each n in theFieldNodeList
for each attr in n.attributes
if attr.Value = "name" then
strTSNames = strTSNames & n.text & ";"
end if
next
next
end if
set theFieldNodeList = Nothing
set myRoot = Nothing
end if
if strTSNames <> "" then
strTSNames = left(strTSNames, len(strTSNames) - 1)
end if
RetrieveTSNameByTLFolder = strTSNames
End Function
If we retrieve the TestSets to be executed we can now call the jenkins pipeline passed all the informations and also the Json (AllTheTestSetPath).
JENKINS PIPELINE
The second big step is the entire pipeline that consists of 3 macro steps that are:
1. Client Execution Login
This step is very important because it manage the activation and the login to single client. The infrastructure consist on a "Focal Point" machine from which scan a pool of executors' machine (UFT Client) and selects one to be logged in. Here is a picture that describe the functionality:
To do such job all the machines (Focal Point and Executors) must be visible each others. The user logged in to the FP must have the admin grant to "ask" the system about the status of the others executors. The FP user must run WMI query (all this system is on a Windows environment).
You have first to create your own algorithm to choose the client to verify the availability. You can do this in different mode; one idea could be create a dictionary of machine names and information about username logon for each client. When you create this do a loop for each machine, analyzing its status to understand if it is selectable for login and execute tests.
To verify the availability you have to do some WMI Query
These are some vbscripts' instructions to do so:
a. Retrieve the WMI object to do the query for a specific "clientname":
Dim objWMIService
Set objWMIService = GetObject("winmgmts:{impersonationLevel=impersonate}!\\" & clientname & "\root\cimv2")
b. verify if the client is visible for jenkins
All the machine must be registered on jenkins Master; you can understand if a client is visible to jenkins verifying if the jenkin's process on the client is running:
Const J_SLAVE = "jenkins-slave.exe"
Dim bolJenkins : bolJenkins = false
Dim colItemsJenkSlave : Set colItemsJenkSlave = objWMIService .ExecQuery("Select * from Win32_Process Where Name = '" & J_SLAVE & "'",,48)
on error resume next
For each objItem in colItemsJenkSlave
if err.number <> 0 then
exit for
end if
'if there is no error the jenkin's process is up and running
bolJenkins = true
Next
on error goto 0
.....
bolJenkins is a boolean variable that tell us if the client is visible to jenkins (true) or not (false)
c. verify if someone is logged in to the client
you can verify if someone is logged in through this query:
Const EXPLORER = "explorer.exe"
Dim strWhoIsLoggedIn : strWhoIsLoggedIn = ""
Dim colItems : Set colItems = objWMIService .ExecQuery("Select * from Win32_Process Where Name = '" & EXPLORER & "',,48)
For each objItem in colItems
objItem.GetOwner user, domain
strWhoIsLoggedIn = domain & "\" & user
Next
....
you can analyze strWhoIsLoggedIn variable to understand if someone is logged in.
With the same query of b and c points, if a user is logged in, you can understand if UFT is running and also if the Scheduler is running (process' names: UFT.exe and wexectrl.exe).
So you should be able to retrieve the client that is free and available to execute test. Once you retrieve it you have to logged in. You can use RDP mode using cmdkey and mstsc commands. Here an example on how to connect via RDP (you must know user and password to connect):
First you must tell the system the machine and the user you use to connect to;
using WScript.Shell object you can use the .Run method to run the command:
"cmdkey /generic:TERMSRV/" & clientname & " /user:" & user & " /pass:" & password
Second you have to use again the .Run method of Wscript.Shell to run the RDP Session with this command: "mstsc /v:" & clientname
The connection will work only if in your system is possible to connect via RDP using saved credentials. You can try to do a simple script that try to do so and if you retrieve the credential popup it means that is active a restrictive policy. You must ask to the system admin to remove it for your need.
Note: there are some technical issue about the communication between the FP and UFT Client. You can see details here.
2. TestSet Execution
The execution of the testset can be demanded to the ALM plugin for Jenkins (Micro Focus Application Tools).
You have to set up the ALM environment into Jenkins through the /configure section (http[s]://<JENKINSADDRESS>/configure) into the "Application Lifecycle Management" entry: in particular you have to set the ALM Server Name and the ALM Server URL in the usual format (http://ALMSERVERADDRESS:[port]/qcbin).
You have also to create the credential to use to connect to ALM Projects. To do so go to /credential section (http[s]://<JENKINSADDRESS>/credential) and create the new entry.
When you use the plugin to run TestSet of Default type you have to set all TestSet paths in the form "Root\....\...\TestSetName" (in case of multiple TestSet you have to create a list of paths separated by '\n'). In the pipeline you have to pass the credentials, domain, project and remember to use the RUN_LOCAL mode to the Exec node choosed before. For example here a piece of groovy code to do such:
withCredentials([usernamePassword(credentialsId: 'ALM', passwordVariable: 'ALM_PWD', usernameVariable: 'ALM_USR')]) {
runFromAlmBuilder almUserName: ALM_USR,
almPassword: ALM_PWD,
almDomain: '${ALM_DOMAIN}',
almProject: '${ALM_PRJ}',
almRunHost: '',
almRunMode: 'RUN_LOCAL',
almServerName: 'ALM_ADDR',
almTestSets: TESTPATH,
almTimeout: '3600'
}
The timeout is in seconds.
The idea is to run the pipeline to the Focal Point node but when a machine is choosed this node is set for the local execution, so the RUN_LOCAL is done on the Exec UFT Client and not on the FP machine. With this kind of execution FP will not remotely asked the UFT Client to run testset so the scheduler doesn't run on FP machine.
Note: You can create the pipeline to run testset in sequence mode, choose a client and exec tests, or in parallel mode that means choose and login to more clients, prepare the executions and then shot them through the parallel statement of the groovy.
3. Result Notification
TestSet object in ALM has the "Automation" tab that allow you to activate the Execution Summary section on which you can tell ALM to send e-mail to a specific user. The only problem is that if the Run of the testset is performed by an external subject and not through its UI it doesn't send the execution report. So you need to read yourself these informations.
To do so you can use this vbscript code and insert it into the pipeline as the last step. For a better separation of tasks I suggest to demand this step to the FP node:
....
suppose to have theTestSet object (tdc is the tdconnection object)
Dim ExecNotifSett : set ExecNotifSett = theTestSet.ExecutionReportSettings
if Not(ExecNotifSett.Enabled) then
set ExecNotifSett = Nothing
set theTestSet = Nothing
tdc.disconnect
set tdc = Nothing
wscript.quit 3
end if
Dim lstIstanzeTest : set lstIstanzeTest = theTestSet.TSTestFactory.NewList("")
Dim myBody : myBody = CreateHTMLBody(lstIstanzeTest)
tdc.SendMail _
ExecNotifSett.EMailTo, _
"myMailFrom@example.com", _
"Execution Report - TestSet: " & theTestSet.Name, _
myBody, _
, _
"HTML"
set lstIstanzeTest = Nothing
set ExecNotifSett = Nothing
set theTestSet = Nothing
tdc.disconnect
set tdc = Nothing
wscript.quit 0
'Function to Create the HTML Body: some parameter are passed in the call to the script
'such as DOMAIN, PROJECT, J_LINK (the link to the jenkin's log)
Function CreateHTMLBody(lstTSTest)
Dim todayDate, ddaayy, mooonth, yyeeaarr
Dim actualHH, HHH, mmin
Dim theTS_Name, T_Stat, theBoss, theTesterrr, theExecDate, theExecHour
Dim theDateToFormat
Dim theLastRun
yyeeaarr = year(now)
if day(now) < 10 then
ddaayy = "0" & day(now)
else
ddaayy = day(now)
end if
if month(now) < 10 then
mooonth = "0" & month(now)
else
mooonth = month(now)
end if
todayDate = ddaayy & "/" & mooonth & "/" & yyeeaarr
if hour(now) < 10 then
HHH = "0" & hour(now)
else
HHH = hour(now)
end if
if Minute(now) < 10 then
mmin = "0" & Minute(now)
else
mmin = Minute(now)
end if
actualHH = HHH & ":" & mmin
Const COLOR_PASSED = "#58BA2A"
Const COLOR_FAILED = "#ff3333"
Const COLOR_NORUN = "#7bb7ed"
Const COLOR_NCOMP = "#ffaf04"
Const COLOR_BLACK = "#000000"
Dim strRes : strRes = ""
Dim theColour : theColour = COLOR_BLACK
strRes = "<html>" & vbNewLine & vbTab & _
"<head>" & vbNewLine & vbTab & vbTab & _
"<meta http-equiv=" & chr(34) & "Content-Type" & chr(34) & " content=" & chr(34) & "text/html; charset=utf-8" & chr(34) & ">" & vbNewLine & _
"</head>" & vbNewLine & _
"<body>" & vbNewLine & vbTab & _
"<style>" & vbNewLine & vbTab & _
"h2 {font-family:'verdana' , 'arial' , 'helvetica' , sans-serif;font-size:13px;font-weight:bold;color:#8e001d} " & vbNewLine & _
"h3 {font-family:'verdana' , 'arial' , 'helvetica' , sans-serif;font-size:11;font-weight:bold;color:white} " & vbNewLine & _
"tr {font-family:'verdana' , 'arial' , 'helvetica' , sans-serif;font-size:12px;font-weight:normal;color:#000000;vertical-align:top} " & vbNewLine & _
"td {font-family:'verdana' , 'arial' , 'helvetica' , sans-serif;font-size:11px;font-weight:normal;color:#000000;border:0 solid dimgray} " & vbNewLine & _
"hr {font-family:'verdana' , 'arial' , 'helvetica' , sans-serif;font-size:12px;font-weight:normal;color:#a3a9b1} " & vbNewLine & _
"body {font-family:'verdana' , 'arial' , 'helvetica' , sans-serif;font-size:11px;font-weight:normal;color:#000000} " & vbNewLine & _
"table {font-family:'verdana' , 'arial' , 'helvetica' , sans-serif;font-size:11px;font-weight:normal;color:#000000;border:0 solid dimgray} " & vbNewLine & _
"</style>" & vbNewLine & _
"<p></p>" & vbNewLine & _
"<p><b>Domain: </b><font color=" & chr(34) & COLOR_BLACK & chr(34) & ">" & DOMAIN & "</font></p>" & vbNewLine & _
"<p><b>Project: </b><font color=" & chr(34) & COLOR_BLACK & chr(34) & ">" & PROJECT & "</font></p>" & vbNewLine & _
"<p><b>Test set: </b><font color=" & chr(34) & COLOR_BLACK & chr(34) & ">" & TESTSET & "</font></p>" & vbNewLine & _
"<p> </p>" & vbNewLine & _
"<p><b>Completion Date: </b><font color=" & chr(34) & COLOR_BLACK & chr(34) & ">" & todayDate & "</font></p>" & vbNewLine & _
"<p><b>Completion Time: </b><font color=" & chr(34) & COLOR_BLACK & chr(34) & ">" & actualHH & "</font></p>" & vbNewLine & _
"<p><b>Details: </b><a href=" & chr(34) & J_LINK & chr(34) & ">Pipeline Execution Log</a></p>" & vbNewLine & _
"<p> </p>" & vbNewLine & _
"<p>" & vbNewLine & _
"<title>Execution Report</title>" & vbNewLine & _
"</p>" & vbNewLine & _
"<div style=" & chr(34) & "border:0 hidden #f3f3f3" & chr(34) & ">" & vbNewLine & _
"<center>" & vbNewLine & vbTab & _
"<h2>Execution Report</h2>" & vbNewLine & _
"</center>" & vbNewLine & _
"<table width=" & chr(34) & "100%" & chr(34) & "align=" & chr(34) & "center" & chr(34) & ">" & vbNewLine & vbTab & _
" <tbody>" & vbNewLine & vbTab & _
" <tr>" & vbNewLine & vbTab & _
" <td style=" & chr(34) & "background-color:white;vertical-align:top;border:solid 1px #8e001d" & chr(34) & ">" & vbNewLine & vbTab & _
" <table width=" & chr(34) & "100%" & chr(34) & "style=" & chr(34) & "background-color:white;padding-top:3px;padding-right:3px;padding-left:4px;vertical-align:top;border-bottom:0px solid #f3f3f3" & chr(34) & ">" & vbNewLine & vbTab & _
" <tbody>" & vbNewLine & vbTab & _
" <tr>" & vbNewLine & vbTab & _
" <td style=" & chr(34) & "color:#000000" & chr(34) & ">Test Name</td>" & vbNewLine & _
" <td style=" & chr(34) & "color:#000000" & chr(34) & ">Execution State</td>" & vbNewLine & _
" <td style=" & chr(34) & "color:#000000" & chr(34) & ">Responsible</td>" & vbNewLine & _
" <td style=" & chr(34) & "color:#000000" & chr(34) & ">Tester</td>" & vbNewLine & _
" <td style=" & chr(34) & "color:#000000" & chr(34) & ">Execution Date</td>" & vbNewLine & _
" <td style=" & chr(34) & "color:#000000" & chr(34) & ">Execution Time</td>" & vbNewLine & _
" </tr>" & vbNewLine
For each inst in lstTSTest
theTS_Name = inst.TestName
T_Stat = inst.Status
theBoss = tdc.TestFactory.Item(inst.TestID).Field("TS_RESPONSIBLE")
theTesterrr = inst.Field("TC_ACTUAL_TESTER")
theExecDate = ""
theExecHour = ""
if T_Stat <> "No Run" then
theDateToFormat = CDate(inst.Field("TC_EXEC_DATE"))
theExecDate = DatePart("d",theDateToFormat) & "/" & DatePart("m",theDateToFormat) & "/" & DatePart("yyyy",theDateToFormat)
theExecHour = inst.Field("TC_EXEC_TIME")
end if
Select Case T_Stat
Case "Passed", "Passed with Minor":
theColour = COLOR_PASSED
Case "Failed":
theColour = COLOR_FAILED
Case "No Run":
theColour = COLOR_NORUN
Case "Not Completed":
theColour = COLOR_NCOMP
Case Else:
theColour = COLOR_BLACK
End Select
strRes = strRes & _
" <tr style=" & chr(34) & chr(34) & ">" & vbNewLine & vbTab & _
" <td>" & theTS_Name & "</td>" & vbNewLine & _
" <td style=" & chr(34) & "color:" & theColour & chr(34) & ">" & T_Stat & "</td>" & vbNewLine & _
" <td>" & theBoss & "</td>" & vbNewLine & _
" <td>" & theTesterrr & "</td>" & vbNewLine & _
" <td>" & theExecDate & "</td>" & vbNewLine & _
" <td>" & theExecHour & "</td>" & vbNewLine & _
" </tr>" & vbNewLine
next
strRes = strRes & _
"</tbody>" & vbNewLine & _
"</table>" & vbNewLine & _
"<br>" & vbNewLine & _
"</td>" & vbNewLine & _
"</tr>" & vbNewLine & _
"</tbody>" & vbNewLine & _
"</table>" & vbNewLine & _
"</div>" & vbNewLine & _
"</body>" & vbNewLine & _
"</html>"
CreateHTMLBody = strRes
End Function
This is the REST and Jenkin's solution you could apply in your organization to have an automatic system to run testsets.
There are some other utilities to manage better the machines' allocation such as the reboot of the client to have a clean situation for next test session.
This is only an idea on how to deal with the integration of jenkins.
Questo sito è stato realizzato con Jimdo! Registra il tuo sito gratis su https://it.jimdo.com