Tuesday, May 12, 2009

Top 25 common programming bugs every tester should know

This is the article which I came accros. It's really good and hence I am including this article over here.
Just a quick note to share a useful resource with you. Just came across a good article "25 common programming errors" for software programmers and software testers. Basically this is more useful for programmers but I think software testers can get insight on how developers can unknowingly leave bugs in software programs.
Each bug listed in this resource can lead to serious software vulnerabilities if not fixed. The top 25 security bugs list will help programmers to avoid some common but serious coding mistakes. For software testers list will be useful as a security testing checklist for Internet as well as for testing desktop application.
Here are few top security vulnerabilities discussed in detail in this article:
Improper input validation
Improper escaping of output or encoding
SQL injection
Cross-site scripting
Race conditions
Information leak in error messages
Error while transmitting sensitive information
Memory leak
External control of critical data and file paths
Improper initialization
Improper authorization
Client side security checks
I think, the most common security vulnerability mistake developers make is "Client side enforcement of server side security".
Check out below article so that you can at least help developers for improving their code standards
Top 25 common programming Errors

Read more...

Basic tips for testing multi-lingual web sites / applications

These days a number of web sites / applications are deployed in multiple languages. As companies perform more and more business in other countries, the number of such global multi-lingual web applications will continue to increase.
Testing web sites / applications supporting multiple languages has its own fair share of challenges. In this article, I will share seven tips with you that will enable you to test the multi-lingual browser-based applications in a complete way:
Tip # 1 – Prepare and use the required test environment
If a web site is hosted in English and Japanese languages, it is not enough to simply change the default browser language and perform identical tests in both the languages. Depending on its implementation, a web site may figure out the correct language for its interface from the browser language setting, the regional and language settings of the machine, a configuration in the web application or other factors. Therefore, in order to perform a realistic test, it is imperative that the web site be tested from two machines – one with the English operating system and one with the Japanese operating system. You might want to keep the default settings on each machine since many users do not change the default settings on their machines.
Tip # 2 – Acquire correct translations
A native speaker of the language, belonging to the same region as the users, is usually the best resource to provide translations that are accurate in both meaning as well as context. If such a person is not available to provide you the translations of the text, you might have to depend on automated web translations available on web sites / applications like wordreference.com and dictionary.com. It is a good idea to compare automated translations from multiple sources before using them in the test.
Tip # 3 – Get really comfortable with the application
Since you might not know the languages supported by the web site, it is always a good idea for you to be very conversant with the functionality of the web site. Execute the test cases in the English version of the site a number of times. This will help you find your way easily within the other language version. Otherwise, you might have to keep the English version of the site open in another browser in order to figure out how to proceed in the other language version (and this could slow you down).
Tip # 4 – Start with testing the labels
You could start testing the other language version of the web site by first looking at all the labels. Labels are the more static items in the web site. English labels are usually short and translated labels tend to expand. It is important to spot any issues related to label truncation, overlay on/ under other controls, incorrect word wrapping etc. It is even more important to compare the labels with their translations in the other language.
Tip # 5 – Move on to the other controls
Next, you could move on to checking the other controls for correct translations and any user interface issues. It is important that the web site provides correct error messages in the other language. The test should include generating all the error messages. Usually for any text that is not translated, three possibilities exist. The text will be missing or its English equivalent will be present or you will see junk characters in its place.
Tip # 6 – Do test the data
Usually, multi-lingual web sites / applications store the data in the UTF-8 Unicode encoding format. To check the character encoding for your website in mozilla: go to View -> Character Encoding and in IE go to View -> Encoding. Data in different languages can be easily represented in this format. Make sure to check the input data. It should be possible to enter data in the other language in the web site. The data displayed by the web site should be correct. The output data should be compared with its translation.Tip # 7 – Be aware of cultural issues
A challenge in testing multi-lingual web sites / applications is that each language might be meant for users from a particular culture. Many things such as preferred (and not preferred) colors, text direction (this can be left to right, right to left or top to bottom), format of salutations and addresses, measures, currency etc. are different in different cultures. Not only should the other language version of the web site provide correct translations, other elements of the user interface e.g. text direction, currency symbol, date format etc. should also be correct.
As you might have gathered from the tips given above, using the correct test environment and acquiring correct translations is critical in performing a successful test of other language versions of a web site.
It would be interesting to know your experience on testing multi-language web sites / applications.
I worked with I18N (Internationalization) project Development and Testing with 24 languages. Truely speaking, it's an ossom experience to work with such applications. As far as India is concerned, our L10N (Localization) project Development and Testing includes major 7 languages (Hindi, Marathi, Gujarati, English, Kanadda, Telugu, Urdu). What we exactly use in our mobile or windows or any other applications, we need to understand it's functionality. Assume a DTP operator or a designer does mistake in language translaction, how worst it looks. No doubt, L10N and I18N projects are purely for GUI testing. But this finally crashes systems heavily if not handled correctly. For testers giving sign-off or test closure document should always mention the risk of string conversion and testing done is limited and application specific.
You can reach out to me if you need any other information on this.

Read more...

How to apply Testing in Real World Environment


Quite often I receive mails from friends asking for some testing exercises. According to me, if you are alert enough, you can find lots of testing exercises in your day to day life. Can’t agree with me? Read on…Today I got an SMS (Forward of course) from a friend. The content of that message was as follows:"If you are forced to draw money by a robber in an ATM, then just enter your PIN number in reverse order. By doing so, you will be allowed to withdraw money from you’re a/c and at the same time the cop will be informed! So the cop will reach the ATM in a short while and rescue you."At first sight, this might seem a very useful message. But a tester is always trained and taught to be skeptical about everything. And I am no exception. So how could I take this piece of information as true, without making further observations/investigations?So I put on my tester’s shoes and tried to analyze it. And here are my observations:1. If this was true, then I should have known this before. Because, if it was true, the bank should have informed me about this when I created my a/c and was given my ATM card. How could they miss to transfer such an important instruction?
2. There are hundreds of banks world over. But this SMS never told about the bank which provides this facility. That meant this information was surely incomplete (if not incorrect).
3. Now coming to the loose link of the message. At some point, the SMS tells about entering reverse PIN number in order to activate some security system. At first sight, this sounds like a brilliant method. Isn’t it? But just think for a while, and you will know this can’t be right. If this was true, then how about the PIN numbers like 1001, 2002, 1221, 2332, 1111, 2222 and so on… (Palindromic Numbers) (these are my test data). If my PIN is one of those palindromes, then how to activate that security mechanism? Then I thought one work around for this is to disallow Palindromic numbers as your PIN. But the idea itself sounded stupid. Simply because, there are lots of Palindromic numbers within 9999 (the maximum possible PIN Number). And I have never seen a message in an ATM machine restricting me from using a Palindromic number as my PIN. But I did not want to believe that argument of mine, without actually seeing (executing my test case with my pre-set test data) it. So I immediately rushed to my nearest ATM counter and tested this. And I found that there is no such restriction for such numbers (test case passed!). Then I checked the same test with two other bank a/c ATMs (regression testing!). And as expected here also my test cases passed! This test almost made me sure about the inaccuracy of the SMS message.
4. Just as some additional arguments to strengthen my point, I again looked at the SMS again. And there it is. If at all we accept this message to be true, still then do you think that "the cop will reach the ATM in a short while and rescue you", keeping in mind that this is India?There are still lots of information left in the message which prove that the information is a hoax. So I would like to leave them for my readers and would like to see, how they use their testing skills to find them out.Hints: Always use the 3 basic weapons of a tester. i.e. Observe, Analyze and be Skeptical.There are lots of testing exercises lying around loosely in your own life too. Try to identify them and try to test them using your very own testing skills.Happy Testing…

Read more...

Emotions and Software Testing!


"Software Testing is a skill. While this may come as a surprise to some people it is a simple fact." - (Fewster, Graham: "Software Test Automation")
Recently I was talking with a fellow software tester and he queried – "Debasis, what is the role of human (can be a software programmer, a tester, etc) emotions in software defects? Do you feel that, to become a good tester we should leave aside our emotions or rather use our emotions to find more defects in the software under test?"
This question made me to think over the role of emotions in software development and in software testing and here are my thoughts on the matter!
Regardless of the roles and designation, whoever is working on the software development team is a human being! And I believe every human being is vulnerable to emotions! Emotions are integral part of human psychology and we just can’t leave our emotions even if we want to! As we can’t get rid of emotions, the next question is how can it affect the quality of software!
» Can people’s (System Analysts, Designers, Programmers, Technical Writers, and even Testers) emotional state at a given time contribute to defective software?
» Can people’s moods or emotions play a role in software quality?
» Can software quality play with people’s (Client, End Users) emotions?
» Can people’s (Testers) emotions be used to enhance the software quality?
Assuming the answer to all of the above 4 questions to be "Yes" (of course depending on the particular context), I think I can assert that:
» Under certain contexts, human emotions may be responsible for defects in the software.» Some defects can be capable of playing with human emotions.» Some defects can be detected using the emotional response of a tester.
Depending on this I have tried to classify defects/bugs, broadly into 3 categories:» Defects/Bugs that are created due to human emotions. [Type A]
» Defects/Bugs that play with human emotions. [Type B]
» Defects/Bugs that are detected by human emotions. [Type C]
Type A: Once I was assigned to test a B2B (Business-to-Business) web portal. We had a pretty good developer team to work with. We (the testing team) were system testing the final candidate, which was scheduled for a release unless any showstopper defect/bug was found! I was regression testing a module, which had undergone some recent code refactoring! [Note: Refactoring neither fixes bugs nor adds new functionality. Rather it improves the understandability of the code or changes its internal structure and design, and removes dead code, to make it easier for human maintenance in the future.] And I got an out of memory error! This error was not there when I had tested it last time. So at first glance this error looked like an error resulted due to the recent refactoring in the source code! When it was intimated to the programmer who had refactored that module, he was puzzled! I knew him to be a good programmer and he proved me right by identifying the root of that error very quickly! It was due to a method that was recursively calling itself and was forcing the program to get stuck in a loop, thus making a "new" object each time round! But the programmer was very much upset with himself for making such a silly (according to him) mistake. Usually novice programmers make these kinds of mistakes and he knew he would not have done it in steady mental state! He was trying to figure out how he could do such a stupid mistake and then remembered that one of his close relative had died last week and he was very much heart broken and stressed. It was when he was asked and had to do that code refactoring. Could that error have been as a result of his disturbed emotional state? Is this a case where a disturbed emotional state is capable of producing defective software?
Type B:
We come across such situations quite often in our day-to-day life! Don’t you find yourself highly irritated when the printer (may be the driver software that runs the printer) just refuses to print your bill and you end up standing at the cash counter of the shopping mall even after having paid your bill amount? Don’t you get frustrated when you see a message popping up (this happens after you have completed all the initial steps and are waiting for your cash) on the ATM machine telling that there is no cash to be delivered at the moment? [Note: If it was saying about not sufficient cash that would have been understandable. What if you get the message even for the minimum allowed withdrawal amount?] Don’t you start muttering, "Why can’t they just show me this message at the very beginning? If they know there is no cash to be dispensed, why are they wasting my time by leading me through the successive steps before displaying the message finally?" Well, I am not sure if you get frustrated/irritated when you face such situations. But as an end user I do get terribly annoyed in such cases. Can we classify these errors/defects as those, which play with human emotions?Type C:
Once I was testing a web portal. It was already tested by another group of testers and I was supposed to just run-through the site as a final sweep of testing. The look and feel of the web portal was just great. After taking a quick look at the Guest User module (which was apparently clean without any major hiccups) when I started testing the Registered User module, the Login Screen just froze (well, it looked so) as I tried to login using a valid login credential! More than 30 seconds must have passed when I started getting impatient and confused! It was quite frustrating to see a web portal (which was supposed to take over 500 concurrent hits) freeze at the login screen before even I (a Registered User) could take a look at the portal! And then, after almost 45-50 seconds the login page finally took me to the next page (Welcome Screen for the Registered User showing me as Logged On). After that every thing seemed to work fine again; there were no more hangs, no crashes, no error messages and the system behaved like a good-obedient-boy. But to me, the long delay of the login procedure appeared like a serious issue from the end user perspective and I went ahead to log that as a High Severity defect in our defect tracker. Later it was discovered that the dynamic load balancing mechanism was having conflict with the distributed web servers and thus resulting in the unwelcome delay! I am wondering if I could have still sensed this as a potential problem and logged it as a defect unless my emotions (impatience, confusion, frustration, annoyance, irritation etc) had not guided me at that instance!Wikipedia defines an emotion as a "complex reaction pattern, involving experiential, behavioral, and physiological elements, by which the individual attempts to deal with a personally significant matter of event." It arises without conscious effort and is either positive or negative in its valence.Emotions, if used correctly can be very powerful in finding defects in software. Emotions such as - stress, anxiety, depression, frustration, confusion, annoyance, impatience, boredom, irritation, curiosity, amusement etc can be good at pointing towards possible defects in software while testing. I have noticed that defects relating to performance issues, usability issues, reliability issues and of course functionality issues are easier to be detected using emotions as your guiding force.» Do you also believe in emotion-guided-testing?
» Do you think that human emotions play important role in the quality of software?» Have you ever come across some similar experience(s) where you noticed human emotions playing roles in defect life cycle (introduction, detection or affecting the end user)?
I am eagerly waiting to hear your stories. Feel free to share your views/ideas on this topic by commenting.
Happy Testing… Further Readings:
Emotions and Test Oracles [Power Point Slide] - By Michael Bolton, on how it might be difficult to automate human emotions while testing! And how human emotions can be the biggest factor in recognizing or interpreting defects/bugs when we see them for the first time.
Bugs and Emotions – By Pradeep Soundararajan, on how defects/bugs are capable of playing with human emotions! An interesting anecdote of a bug that he experienced after registering with a matrimonial website!

Read more...

Friday, May 8, 2009

Website Testing - Did you miss anything while Testing?


A lot has been written and discussed about Website Testing till date. But still, website testing is probably one of the most commonly confused topic among testers! Need evidence? Spend a little time searching and you can see tons of queries flooding the Internet (Online Forums, Usenet Groups, Orkut Communities, Tech Corners etc) regarding website testing, how a website should be tested, what should be tested, which things should be given priority while testing, what should not be given much importance while testing and so on. I might well be the trillionth person on this planet to write an article on Website Testing here! But I am writing this because I am really tired of replying emails asking me to write an article on Website Testing in my blog. However, this article is not an attempt to show you how you should test your website. You should understand that there is no universal rule to testing that can fit all kind of similar testing assignments. The testing approach that is going to follow consists of a checklist of items that might be tested while testing a website. This does not mean that following this checklist can essentially guarantee you success while testing *any and every* kind of website. In contrary, this post is meant to be used as a website testing cheat-sheet, a kind of checklist of items to remember while testing a website! While following this checklist, the chance of success (or failure) greatly depends on your own particular context! However, here is the cheat-sheet for website testing:
[A] Functionality Testing:
While testing the functionality of the websites the following areas should be tested.
a) Links/URL Testing: There are mainly 4 types of links in most websites.
» Internal links [Test the links that point to the pages of the same website.]
» External links [Test the links that point to external websites.]
» Mail links [Test if the email links open the default email client with the recipient email ID already filled in the "To" field.]
» Broken links [Test if any of those links are broken or dead! Free tools like Link Valet (which is convenient for checking a few pages) or Xenulink (convenient for checking a whole site) can be helpful in testing broken links.]
b) Forms Testing: The web forms should be consistent and should contain all the required input and output controls. Test the integrity of the web forms and the consistency of the variables.
c) Validation Testing:
» You can use tools like W3C validator to test and make sure that you have valid HTML (or XHTML).
» Most of the modern day websites use CSS (Cascading Style Sheet). You can use tools like W3C CSS validator to test and validate the CSS used in your site.
» Test the different fields for field level validation. Test and validate user inputs like: TextBox inputs, ComBox inputs, DropDownBox selections, KeyDown, KeyPress, KeyUp etc.
d) Test the Error messages: Error messages are integral part of any well-developed website and they guide the user whenever any wrong/unexpected input is submitted. Testing the Error messages is very important as a badly designed Error message can misguide the end user about the actual impact of the error! About testing error messages, Ben Simo has a nice article that you might find interesting.
e) Testing optional and mandatory fields: Test if the web forms handle the optional and mandatory fields efficiently. Ideally, the application should not allow you to proceed unless you have filled in ALL the mandatory fields and should not restrict you from proceeding if you have left ANY of those optional fields unfilled!
f) Database Testing: Most of the modern day websites come with a backend database (unless it’s a site consisting of purely static web pages). So testing the database for its integrity becomes essential to make sure the website is able to handle the data processing effectively.
g) Cookies Testing: A cookie is information that a website (server side) puts on your hard disk (client side) so that it can remember something about you at a later time. If your website sets cookies at the client machines then test to check that cookies and other session tokens are created in a secure and unpredictable way. Poor handling of cookies can result in security holes and vulnerabilities that can be taken advantage by malicious users and hackers. Here is a nice article on Testing for Cookie and Session Token Manipulation.
h) Client-side Testing: Test the temporary Internet files on the client side system to make sure if any sensitive data (like password, credit card number etc) is being stored in the client system without being encrypted or in an unsecured way.[B] Performance Testing:
IEEE defines Performance testing as the testing conducted to evaluate the compliance of a system or component with specified performance requirements. The goal of performance testing is not to find bugs, but to eliminate bottlenecks and establish a baseline for future load/stress testing. Performance testing can be applied to understand the website’s scalability, any loopholes in the load balancing and to test the response time between a request (from the client) and the reply (from the server) and the amount of load/stress the site is able to sustain. Scott Barber’s PerfTestPlus and Chris Loosley’s Web Performance Matters are two great resources for more exhaustive information on Performance Testing.
[C] Connection Speed Testing:
Test the website over various Networks like - Dial up connection with a 56-kbps modem (hard to believe, but there are lot of web users who still use a modem), ISDN, cable connection, broadband connection with different download speeds, DSL connection, satellite internet etc. With slower connection speed, if your website results in slow performance or partial loading of web pages then it should be cause of concern. As testers, we act as the representative of the end users. If we find out that even a part of our intended end user community is going to have problem with the application we are testing, it should be sufficient to raise the issue as a defect!
[D] Web Usability Testing:
If you develop a website which is not user-friendly and is difficult to learn, understand and navigate, then it won’t be of much use for the user. For example, if your website relies way too much on JavaScript for navigation, a browser with disabled JavaScript can render the site unusable. Some of the criteria to keep in mind while testing the usability of a website are:
» Ease of learning (How intuitive and self-explanatory the site is).
» Navigation.
» User satisfaction.
» Web accessibility testing (If all the content and parts of the site are accessible).
» General appearance (Look and feel).
[E] Testing Client-Server Interface:
In web testing the server side interface is tested to verify that communication is properly established with the client. In case of n-Tier architecture based web applications, the middle-tier Business Logic APIs should also be tested (even if they are third-party shrink-wrapped softwares) for any communication/interface malfunctioning![F] Compatibility Testing:
Test your website to verify that it’s pages render adequately in different browsers (e.g. IE 5, IE 6, IE 7, Firefox 2, Opera, Safari etc) using different operating systems (Win XP, Vista, Linux, Mac etc) on different hardware platforms. Different versions, configurations, display resolutions, and Internet connect speeds all can impact the behavior of the web pages and can introduce embarrassing (and often costly) bugs. However, an important thing to remember while testing the web compatibility is – we should first identify the major customer base for the website and then decide the main browsers and OS to test for. Some typical compatibility tests include testing your application:
» Using various browsers.
» Using various window sizes.
» Using various font sizes.
» Using browsers with CSS, JavaScript turned OFF!
» Using browsers with pop-up blockers!
» On various Operating Systems.
» With different screen resolutions and color depths.
» On various client hardware configurations.
» Using different memory sizes and hard drive space.
» In different network environments.
» Ability to take printouts with different printers (Printer-friendly Versions)[G] Web Security Testing:
Often Web Security Testing is also referred to as Penetration Testing. The primary objective for testing the security of a website is to identify potential vulnerabilities/security holes and to patch/repair them. For example, if your website allows some files to be uploaded, your web server should have proper automated antivirus checking in place to detect and disable any attempt of virus uploading by the client side. Some of the major aspects of web security testing are:
» Network Scanning.
» Vulnerability Scanning.
» Password Cracking.
» Log Review.
» Integrity Checkers.
» Virus Detection.
Hope this article helps in giving a rough coverage of areas that needs attention while testing a website. Did I miss something here that should have been included in the checklist? Feel free to let me know by commenting and I will update this post along with proper credit to you.
Update: An important aspect of website testing that I had missed to include in my cheat-sheet is Testability Testing.

Read more...