TASTE bugtrack - Taste
View Issue Details
0000950Taste[All Projects] ASN.1 Compiler v4public2021-08-30 16:272021-09-16 12:43
gmamais 
gmamais 
normalminoralways
closedfixed 
0000950: [FAQAS] Progress on test generation for the ASN.1 case study
Dear Thanassis, We have applied our mutation analysis framework to the auto-generated code produced by the ASN1CC compiler, particularly the auto-generated code has been produced with the ASN.1 grammar you've shared with us during the last year. The analysis was done in two phases, that is, phase (1) mutation analysis, and phase (2) test generation. During phase (1), we applied mutation analysis to the auto-generated code (i.e., the test.c source code), and we generated 5.323 code-driven mutants, that is, 5.323 syntactic altered versions of the original code. Through its test suite, we identified 3.104 mutants, but 2.219 mutants remain undetected, giving a mutation score of 58.13%. During phase (2), we applied test generation to the 2.219 live mutants, and we produced test inputs that killed mutants for 1.717 of them. Considering this augmented test suite, the mutation score is now 90.57%. Now, we started analyzing on the details the content of these 1.717 tests, but we noticed something peculiar about one of them, that I'll describe in the following: The original mutated code is the following one and concerns the function T_INT_IsConstraintValid: flag T_INT_IsConstraintValid(const T_INT* pVal, int* pErrCode) { flag ret = TRUE; (void)pVal; ret = ((*(pVal)) <= 50UL); *pErrCode = ret ? 0 : ERR_T_INT; // original statement *pErrCode = ret ? -1 : ERR_T_INT; // mutated statement return ret; } Our test generation tool tells us that for this case, an input value that kills the mutant is pVal=0. We found this a bit strange, since the original test suite already considers pVal=0 as an input (it also considers pVal=50). To be sure, we also checked the test source code that invokes T_INT_IsConstraintValid: flag T_INT_enc_dec(const T_INT* pVal, int* pErrCode, const char* filename) { static T_INT decodedPDU; flag ret = TRUE; ... // validate decoded data ret = T_INT_IsConstraintValid(&decodedPDU, pErrCode); // pErrCode seems to be never checked if (ret) { ret = T_INT_Equal(pVal, &decodedPDU); *pErrCode = ret ? 0 : 4; if (ret) { char buf[1024]; strcpy(buf, filename); FILE* fp = fopen(strcat(buf,".dat"), "wb"); fwrite(bitStrm.buf, 1, bitStrm.count, fp); fclose(fp); } } ... } And it seems that after the invocation of T_INT_IsConstraintValid, the value of pErrCode is never checked, and probably this is the reason why the mutant that modifies the value of pErrCode is not detected. To summarize, the test suite lacks a verification of the value contained by pErrCode (i.e., it shall check that it does not contain an error code). Is our conclusion correct? We will progress with analyzing the rest of the generated tests but first we would like to have some feedback from your side. Thanks in advance, –– Oscar Cornejo, PhD Research Associate – SVV group SnT – Interdisciplinary Centre for Security, Reliability and Trust UNIVERSITÉ DU LUXEMBOURG CAMPUS KIRCHBERG 29, avenue John F. Kennedy L-1855 Luxembourg Kirchberg T +352 46 66 44 5376 | oscar.cornejo@uni.lu mail from Maxime Maxime.Perrotin@esa.int 12:50 μ.μ. (πριν από 6 ώρες) προς Thanassis.Tsiodras, εγώ Hi both, I believe that the "issue" is here: errCode has been mutated inside IsConstraintValid, and then set again to 0 without an intermediate check, so indeed the mutation is not detected. The point is that the "if(ret)" is the same condition that set errCode to the mutated value inside IsConstraintValid So instead of if (ret), we should/could have if (0 == errCode && ret) Kind regards Maxime ESA - European Space Agency Maxime Perrotin Proba-3 Plaftorm & Payload Software Engineer TASTE Lead Engineer System, Software and Technology Department ESTEC Keplerlaan 1, PO Box 299 NL-2200 AG Noordwijk, The Netherlands maxime.perrotin@esa.int | www.esa.int T +31 71 565 4923 | F +31 71 565 5420
No tags attached.
Issue History
2021-08-30 16:27gmamaisNew Issue
2021-08-30 16:27gmamaisStatusnew => assigned
2021-08-30 16:27gmamaisAssigned To => gmamais
2021-08-30 16:28gmamaisNote Added: 0003796
2021-08-30 16:28gmamaisStatusassigned => resolved
2021-08-30 16:28gmamaisResolutionopen => fixed
2021-09-16 12:43maximeNote Added: 0003799
2021-09-16 12:43maximeStatusresolved => closed

Notes
(0003796)
gmamais   
2021-08-30 16:28   
Fixed according to Maxime proposal.
(0003799)
maxime   
2021-09-16 12:43   
Thanks! closing