Como NÃO testar o seu projeto de Software. DevDay 2014
-
Upload
alexandre-freire -
Category
Technology
-
view
921 -
download
1
description
Transcript of Como NÃO testar o seu projeto de Software. DevDay 2014
Como NÃO testar SW?
uma palestra da
@freire_da_silva
• Director of Product Safety @IndustrialLogic
• Agil desde 2001, lean desde a 1ª série
• Founder: AgilCoop, AgilBits e ).(
• Mestrado na USP: “Reflexões sobre o Ensino de Metodologias Ágeis na Academia, Indústria e Governo”
• Gosto de nadar e construir casas
• Experiência e sucesso com muitos amigos: [email protected]
Você TESTA?
TODO SW TEM BUG?
TODO SW TEM BUG!
/1947
1999: !Imperial instead of metric units !cause Mars Climate Orbiter…
to disintegrate!
if((err=SSLFreeBuffer(&hashCtx))!=0) goto fail; if((err=ReadyHash(&SHA1,&hashCtx))!=0) goto fail; if((err=SHA1.update(&hashCtx,&rand))!=0) goto fail; goto fail; if((err=SHA1.update(&hashCtx,&sign))!=0) goto fail; if((err=SHA1.final(&hashCtx,&hsOut))!=0) goto fail; err = sslRawVerify(…);
sslKeyExchange.c
if((err=SSLFreeBuffer(&hashCtx))!=0) goto fail; if((err=ReadyHash(&SHA1,&hashCtx))!=0) goto fail; if((err=SHA1.update(&hashCtx,&rand))!=0) goto fail; goto fail; if((err=SHA1.update(&hashCtx,&sign))!=0) goto fail; if((err=SHA1.final(&hashCtx,&hsOut))!=0) goto fail; err = sslRawVerify(…);
sslKeyExchange.c
/* Read type & payload length first */
hbtype = *p++; n2s(p,payload); p1 = p; !/* Enter response type, length & copy payload */
*bp++ = TLS1_HB_RESPONSE; s2n(payload, bp); memcpy(bp, p1, payload);
/* Read type & payload length first */
hbtype = *p++; n2s(p,payload); if (1+2+payload+16 > s->s3->rrec.length) return 0; /* silently discard per RFC 6520 sec. 4 */ p1 = p; !/* Enter response type, length & copy payload */
*bp++ = TLS1_HB_RESPONSE; s2n(payload, bp); memcpy(bp, p1, payload);
Você AUTOMATIZA SEUS TESTES?
?
#1e5d91
#1e5d91
1describe “test” do 2 it “should do nothing” do 3 fail 4 end 5 end
1describe “test” do 2 it “should do nothing” do 3 fail 4 end 5 end
1describe “test” do 2 it “should do nothing” do 3 true 4 end 5 end
1describe “test” do 2 it “should do nothing” do 3 true 4 end 5 end ?
NON-FUNCTIONAL REQUIREMENTS
SCALABILITY
RELIABILITY
SECURITYUSABILITY
ACCESSIBILITY
MAINTAINABILITY
#1e5d91
por James Reason
RISCOS
FALHAS
Você APAGA seus testes?
?
100% COVERED CODE.
OF CODE YOU COVERLINE
60@Test 61public void setPrice() { 62 Item item = new 63 BasicItem(0.0, true); 64 65 assertEquals(0.0, 66 item.getPrice()); 67 68 //set new price 69 item.setPrice(1.0); 70 assertEquals(1.0, 71 item.getPrice()); 72}
74@Test 75public void isImported_true(){ 76 Item item = new 77 BasicItem(0.0, true); 78 79 assertTrue(item.isImported()); 80}
74@Test 75public void isImported_false(){ 76 Item item = new 77 BasicItem(0.0, false); 78 79 assertFalse(item.isImported()); 80}
28public Double getPrice(){ 25 return price; 26} 27 28public boolean isImported(){ 29 return imported; 30}
WHY DO WE TEST?
1while true do 2 print “vou parar?” 3end
!Murphy’s law
PAIN
FLOW
Cost of Quality Assurance (QA)
Analysis Design Coding Unit Tests Acceptance Tests Production
freqüencia de Uso Das Funcionalidades
RARO 19%
Nunca 45%
Algumas VEZES 16%
Frequente 13%
Sempre 7%
Pair Programming
?
feedback vs
noise
Speed Robustness
non-deterministic
flakycode
Test failed 21 -mes locally & 36 -mes in dev build in the last 6 months.
@google If they fail we simply run flaky tests 3x (and keep statistics). Developer time is much more valuable than server time.
1@Test 2public void errorExerciseUploadTooBig { 3 goToExerciseIntroPage(); 4 goToNextPage(); 5 6 type(“labArchive”, path() + TOO_BIG); 7 pause(1000L); 8 assertText(“error: Upload too big!”); 9}
sleep() demais
How much QUALITY is ENOUGH?
Bugs/1KLOC
0
1
3
4
5
Indústria Nasa
0,004
Cost($/LOC)
0
225
450
675
900
Indústria Nasa
5
850
It will only run once
VS
<- costumer facing
back office ->
@martinfowler you’re doing enough testing if the following is true: ■You rarely get bugs that escape
into production ■You are rarely hesitant to change
some code for fear it will cause production bugs
@joshuakerievsky Test first/after misses the point that TDD is more about emergent design than it is about testing. Do you practice emergent design?
@kentbeck If I don’t typically make a mistake(...), I don’t test for it. !
Wish there were more examples of “what not to test and how not to test”.
2def create_name(fname, lname): 3 if not isInstance(fname, basestring): 4 raise TypeError(“fname must be a String”) 5 if not isInstance(lname, basestring): 6 raise TypeError(“lname must be a String”) 7 name = fname + lname
1 require ‘spec_helper‘ 2 describe Candidate do 3 context ‘associations‘ do 4 it { should have_many(:proposals) } 5 end 6 7 context ‘validations‘ do 8 it { should validate_presence_of :name } 9 10 it { should ensure_lenght_of(:phone). 11 is_at_least(7). 12 is_at_most(14) 13 } 14 15 it { should_not 16 allow_value(‘blah‘).for(:site) } 17 18 it { should 19 allow_value(‘http://www.blah.com‘) 20 .for(:site) } 21 end 22end
1 require ‘valid_url‘ 2 class Candidate < ActiveRecord::Base 3 has_many :proposals 4 5 validates :name, presence: true 6 7 validates :phone, :length => 8 {:in => 8..14}, 9 :allow_blank 10 11 validates :site, :url => true, 12 :allow_nil => true 13 14end
1 require ‘spec_helper‘ 2 describe Candidates do 3 4 let(:candidate) {double ‘candidate‘} 5 6 before :each do 7 Candidate 8 .should_receive(:find) 9 .with(1).and_return(candidate) 10 end 11 12 it ‘should find the candidate‘ do 13 c = Candidate.find(1) 14 c.should eql candidate 15 end 16end
1Feature: Create proposal 2 As a candidate 3 I want to post my proposals 4 So that voters can evaluate them 5 6 Scenario: 7 Given I am logged in 8 And I am posting a proposal 9 When 10 I fill all fields of the proposal 11 Then 12 I should see a success message
1Scenario: Client sees tooltip for plan 2 Given 3 I select the ‘light‘ plan 4 When 5 I mouse over ‘tooltip‘ 6 Then 7 I should see ‘tooltip‘ content 8 And 9 I mouse out ‘tooltip‘ 10 Then 11 I should not see ‘tooltip‘ content
1require ‘spec_helper‘ 2describe ShoppingCart do 3 4 let(:user) {double ‘user‘} 5 let(:product) {double ‘product‘} 6 7 before :each do 8 Authenticator.should_receive(:auth) 9 .and_return(true) 10 end 11 12 it ‘should addProduct & getTotal‘ do 13 Authenticator.auth(user) 14 cart = ShoppingCart.new 15 cart.add_product(product) 16 cart.get_total 17 end 18end
50@Test 51public void changeMarks() { 52 bot.leftClickAt(view, 53 800, 508); 54 addMarkAt(‘drama’, 1); 55 56 bot.leftClickAt(view, 57 900, 508); 58 addMarkAt(‘act’, 3); 59 60 bot.verifyTooltipAt(30, 190); 61}
PIXEL NAZI
1 require ‘spec_helper‘ 2 describe AddressController do 3 4 it ‘should calculate shipping‘ do 5 get :shipping, :zipcode => ‘90210‘ 6 assigns(:shipping).should == ‘8.2‘ 7 end 8 9 end
external dependencies:
DRY
WET
fixtures considered harmful?
JAVASCRIPT?
module('MultiSelectQuizTests',{ setup: function() { var container = document.getElementById("qunit-fixture"); var question = "Which foods are Mexican?"; var answers = [ { 'answerText': 'Tacos', 'value': true }, { 'answerText': 'Sushi', 'value': false } ]; ! this.myQuiz = new MultiSelectQuiz ( container, question, answers ); }, }); !test( "One correct", function() { checkRadio(0, true); checkRadio(1, true); deepEqual(this.myQuiz.grade(), 1, "just 1"); });
CSS?
it(‘centers logo at top of page’, function() { expect(isConTenteCenteredInPage(logo)).toBe(true); expect(elementPicelsFromTopOfPage(logo)).toBe(12); expect(fontSizeOf(logo)).toBe(22); expect(textColorOf(logo)).toBe(WHITE); });
Students(Can’t(Access(Service(
New(Produc5on(Server(
Ok(To(Toggle((Test(Passes(
Immune(System(Fails(
No(Auto?(Rollback(
No(SMS(
No(Policy(
Non?Standard(Tomcat(Runner(
Nginx(Points(To(Down(Service(
Non?Standard(Java(Version( JRE(Crash(
No(Policy(
And$
And$ And$ And$
análise de uma falha
Test journeys
NÃO É BUG É FEATURE
JAVASCRIPT
80% só até amanhã!código de desconto: DEVDAY2014
HTTP://industriallogic.com/shop
PYTHON