If it ain't broke - You're not trying hard enough!
Google's own blog about stuff in the testing phase... enjoy!
-
Code coverage goal: 80% and no less!
by Alberto Savoia I first posted this article a few years ago on the Artima Developer website; but the question of what's adequate code coverage keeps coming up, so I thought it was time for a repost of Testivus wisdom on the subject.
Testivus on Test Coverage Early one morning, a young programmer asked the great master: “I am ready to write some unit tests. What code coverage should I aim for?” The great master replied: “Don’t worry about coverage, just write some good tests.” The young programmer smiled, bowed, and left. 
Later that day, a second programmer asked the same question. The great master pointed at a pot of boiling water and said: “How many grains of rice should I put in that pot?” The programmer, looking puzzled, replied: “How can I possibly tell you? It depends on how many people you need to feed, how hungry they are, what other food you are serving, how much rice you have available, and so on.” “Exactly,” said the great master. The second programmer smiled, bowed, and left. 
Toward the end of the day, a third programmer came and asked the same question about code coverage. “Eighty percent and no less!” Replied the master in a stern voice, pounding his fist on the table. The third programmer smiled, bowed, and left. 
After this last reply, a young apprentice approached the great master: “Great master, today I overheard you answer the same question about code coverage with three different answers. Why?” The great master stood up from his chair: “Come get some fresh tea with me and...
-
There, but for the grace of testing, go I
By James A. Whittaker
I've had more than a few emails about "antenna-gate" asking me to comment and suggesting clever, stabbing rebukes to a fallen competitor. I might aim a few of those at my own team in the future, some were genuinely funny, but none of them will appear here. Instead I offer first a word of caution and second a reflection that my Mom used to intone whenever disaster occurred around her. It's called "counting your blessings."
First, a caution that those of us who live in glass houses really should keep stones at arms length. The only way anyone can rebuke Apple, without risk of waking up one morning sucking on their own foot, is if they write no software or have no users. Apple does a lot of the former and they enjoy many of the latter. Bugs like this make me sick when they are mine and nervous when they aren't. If any tester in the industry isn't taking stock right now then they either aren't producing any software or aren't in possession of any users, at least ones they wish to keep.
Second, taking stock has made me realize that I enjoy some important blessings that make the infinite task of testing so much more manageable. Indeed, the three blessings I count here are really the reason that testing doesn't fail more often than it does.
The Blessing of Unit Testing
I am thankful for early cycle testing thinning out the bug herd. In late cycle testing major bugs are often masked by minor bugs and too many of the latter can hamper the search for the former. Every bug that requires a bug report means lost time. There is the time spent to find...
-
Testivus, Testability and Dr. Jekyll and Mr. Hyde
by Alberto Savoia
Note: Apparently, there were lots of downloads of the Testivus booklet and I hit some kind of quota on my personal account. If you have problems with reaching the original link below, please try this new download link or this one.
A major topic at this year's GTAC conference is going to be testability: "We also want to highlight methodologies and tools that can be used to build testability into our products." That's great!
Testability is one of the most important, yet overlooked, attributes of code – and one that is not discussed enough. That's unfortunate, because by the time the issue of testability comes up in a project it's usually too late. As preparation and seeding for GTAC, I though it would be fun and useful to get some discussions on testability going. So here we go, feel free to chime in with your thoughts.
A few years ago, after watching one too many episodes of Kung Fu, I was inspired to write a pretentious and cryptic little booklet about testing called "The Way of Testivus" (PDF).
Testivus addresses the issue of testability in a few places, but I would like to start the discussion with this maxim:
To me, "Think of code and tests as one" is the very foundation of testability. If you don't think about testing as you design and implement your code, you are very likely to make choices that will impair testability when the time comes. This position seemed obvious...
-
Update! GTAC 2010 Keynote Speakers
We are thrilled to announce that GTAC 2010 keynotes are finalized. We are very fortunate to have three world-renowned software testing icons: Robert Binder, Dr. Jeff Offutt and Dr. James Whittaker. Robert, Jeff and James bring to GTAC a powerful combination of practical and theoretical experience to help us address key aspect of this year's theme: Test to Testability. Robert will kick-off Day 1 with a keynote on the critical role of testability. On Day 2, Jeff will share his experiences on Automatic Test Generation from source code and about a technique he invented, Bypass Testing, for black-box testing of web applications. James will close the conference with a keynote focusing on the Test(ing) challenges and how to get ahead of them. More details on the specifics of their talk coming soon!
Here are brief bios of our esteemed speakers:
Robert Binder Robert V. Binder is a software entrepreneur and technologist with over 34 years of systems engineering experience. His 1994 analysis of software testability http://portal.acm.org/citation.cfm?id=184077 has had a continuing influence on research and practice in this area. As principal of System Verification Associates, he leads teams that deliver advanced IT assurance solutions. He was recently awarded a U.S. Patent for a unique approach to model-based testing of mobile systems. He is a member of the Editorial Board of Software Testing, Verification, and Review and internationally recognized as the author of the definitive Testing Object-Oriented Systems: Models, Patterns, and Tools. Robert holds an MS in Electrical Engineering and Computer Science from the University of Illinois at Chicago and a MBA from the University of Chicago. He is an IEEE Senior Member.
Dr. Jeff Offutt Dr. Jeff Offutt is Professor of Software Engineering at George Mason University. He has invented numerous test strategies, published over 120 refereed research papers,...
-
Test Driven Integration
By Philip ZembrodIn an earlier post on trying out TDD I wrote how my mindset while coding changed from fear of bugs in the new code to eager anticipation to see the new code run through and eventually pass the already written tests. Today I want to tell about integrating components by writing integration tests first.In a new project we decided to follow TDD from the start. We happily created components, “testing feature after feature into existence” (a phrase I love; I picked it up from a colleague), hitting a small test coverage of around 90% from the start. Obviously, when it came to integrating the components into a product, the obvious choice was to do that test-driven, too. So how did that go?What I would have done traditionally was select a large enough set of components that, once integrated, should make up something I could play with. Since at least a minimum UI would be needed, plus something that does visible or useful things, preferably both, this something would likely have been largish, integrating quite a few components. With the playing around and tryout, I’d enter debugging, because of course it wouldn’t work at first attempt. The not-too-small number of integrated components would make tracking the cause of failures hard, and anticipating all this while coding, I’d have met the well-known fearful mindset again, slowing me down, as I described in my initial TDD post.How did TDI change this game for me? I realized: With my unit test toolbox that can test any single component, I can also test an integration of 2 components regardless of whether they have a UI or do something visible. That was the key to a truly incremental process of small steps.First, write the test...
-
GTAC: Call for Attendance & Proposals
Google Test Automation Conference (GTAC) 2010Call for Attendance & Proposals We are happy to announce that the application process is now open for Attendance and Proposals for the Fifth Google Test Automation Conference (GTAC), to be held in Hyderabad, India on October 28 - 29th.As in previous years, GTAC is an invitation only conference where we enable sharing of great ideas and active participation to challenge and refine our thoughts and experiences. As such the the application process expects you to share your ideas and insights that you would bring to the conference and how these would further the discussion about this year’s theme of Test to Testability. This information will help the committee select a balanced audience of seasoned practitioners, students and academics. Also this year, we are introducing a participant-driven format that will give the power to the attendees to select and voice their opinion on the speakers and the content! To make these changes, we are opening up proposals and attendance applications simultaneously. Once the initial set of participants are finalized, we will conduct online viewing and voting by the participants for presentations. How to applyFor Attendance: Please visit http://www.gtac.biz/call-for-attendance For Proposals (to present): Please visit http://www.gtac.biz/call-for-proposalsDeadlineThe due date for both categories of applications is July 9th, 2010. Registration FeesThere are no registration fees. Please check the FAQ page for more information.Further informationGeneral website: http://www.gtac.biz/Call for proposals: http://www.gtac.biz/call-for-proposalsCall for attendance: http://www.gtac.biz/call-for-attendanceFAQ: http://www.gtac.biz/faqQuestions: Email us at
This e-mail address is being protected from spambots. You need JavaScript enabled to view it
We look forward to your applications and a great GTAC! Finally we would appreciate your help in helping us spread the word about this event.RegardsSujay...
-
Do Know Evil
Web Application Exploits and Defenses by Bruce Leban in Google Kirkland http://google-gruyere.appspot.com/ If you want your application to be as secure as possible, you need to learn how Evil People think. And you'll want to use that knowledge to do penetration testing: attacking your own application to try to find bugs. To help you understand how applications can be attacked and how to protect them from attack, we've created the “Web Application Exploits and Defenses” codelab. The codelab uses Gruyere, a small, cheesy, web application that is full of real world bugs. In the codelab, you'll learn how to: Gruyere is chock full of cool features, and the more features an application has the larger the attack surface. Your application probably has features just like these: Can you match each feature to the vulnerability that it exposes and the exploit it enables?
| Feature | | New template language HTML allowed in snippets File upload capability AJAX Web-based admin console |
-
GTAC!
By James A. Whittaker
Yes I know, I've been quiet. Seriously heads down shipping products and developing what I think are some pretty cool new testing ideas and tools. Perhaps GTAC will be the chance for you to judge that for yourselves. Perhaps it will be worth the wait.
I hope I am invited to speak at GTAC. (Is this an appropriate forum for such lobbying? Should I open it up to a vote on whether I should or should not be there? I am happy not going as India is a long trip, but this is GTAC we are talking about!) There are a number of things that will be ready to either debut or, if I am lucky, open source by then.
Would you like to see a Chrome extension to display bug, test case, coverage and other information as an overlay on top of your web app UI? Imagine being able to see bugs at their exact location on the UI, report bugs by simply right-clicking the errant part of the web page, see footprints where test cases (automated and manual and across multiple testers) have been and lots more useful information. Are you tired of querying databases to see these things and just want your test data as a cellophane wrapper around your web app UI? If you were at STAR this week, you got a preview. But that presentation is already out of date.
Would you like to be able to write automated test cases that can control your web app, your browser and the operating system they are running on? Well if that stack contains Chrome and Chrome OS, you can do it with a new web test framework we are developing. Would you like to do...
-
Google Test Automation Conference 2010 - Hyderabad, India
Thanks for all the enquiries about this year's GTAC event. We have been busy planning and are now ready to share the details. GTAC 2010 will be hosted by Google office in Hyderabad, India on October 28th & 29th, 2010.
As in previous years, the focus of the conference will be on solving software engineering challenges using tools and automation. This year the conference theme is "Test to Testability". With this theme our goal is to look beyond the challenges faced during testing and the complex products that are being developed today. We also want to highlight methodologies and tools that can be used to build testability into our products and to look at how developer tools can be a means towards effective and efficient testing. GTAC 2010 hopes to bring together a group that shares lessons learned and practical experiences regarding testing web apps, services, and systems.
One of the strengths of the conference is that it's driven by a peer group and vocal participation. As in previous years, GTAC is an invitation only conference to share great ideas and to have your thoughts challenged and refined. When you apply, we will be asking you to share with us what ideas and insights you'll be bringing to the conference and how these can further the discussion. The application process for attending will be opening in mid-May 2010.
Today we are also launching the official site for this event and will be updating it and this blog with relevant information in the coming weeks. The next announcement will be a call for attendance and proposals, so do watch these spaces.
Please send suggestions, questions and recommendations to:
This e-mail address is being protected from spambots. You need JavaScript enabled to view it
or post your comments here to this blog.
Looking forward...
-
Google's Innovation Factory (and how testing adapts)
Now that ICST 2010 has concluded, I'm going ahead and posting the keynote paper and the slides. Enjoy! Patrick Copeland
Let me know if you have comments or questions.
Here's a link to the paper. They are semi-complementary.
|
|