Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Doctor Yiff
Jan 2, 2008

GetWellGamers posted:

I'm pretty good at Ad-hoc testing and have taught others how to do it well before, so that shouldn't be a problem. I might check out that book on risk-based testing, Mono, but from what I understand that system's pretty built-in to just haing a priority scale in terms of crash/freeze, critical path, etc. Personally, I think that unless the build is just in shambles you should never assign all your testers to Type1/A/Critical bugs, because while it's true that crash bugs are more important, I'll say in the same breath that in the final product, even simple cosmetic errors are enough to make people stop mid-session and go "Really? Portaling errors? In 2013? Really guys? " It won't stop them from enjoying the game on the same level as a crash or freeze, but it can ruin their enjoyment just as powerfully.

Absolutely agree, and the book goes into why but it's not quite as simple as all that. tl;dr You combine the likelihood of bugs occurring in a system with the user impact if they occur and that informs testing priority. Having a priority scale in your tracker directs the dev's attention, risk analysis directs where you prioritize your efforts, and gives you a logical, easy way to figure out what tests to cut when you inevitably start running out of time, because all the stakeholders know what's important up front.

I'd also try pairing up your more experienced testers with a component of the game and make them the test owner for that system. Introduce them to the dev working on it, see about letting your tester pull that dev's branch locally and test there. You'll catch bugs and fix them a lot faster before they make it into trunk, that tester gets some motivating responsibility and dev face time.

e: Sorry if this is elementary or obvious, just infodumping and spitballing.

Adbot
ADBOT LOVES YOU

GetWellGamers
Apr 11, 2006

The Get-Well Gamers Foundation: Touching Kids Everywhere!
Eeeehhh, I'm a bit leery of letting bottom-level testers check stuff out of the repository, if only for the same reason I only let young relatives "help" put away plastic cups and dishes- if they drop it, it probably won't break.

I do agree with delegating specific testers to areas they're more interested/proficient in- I mentioned it briefly in the "personality spreadsheet" I'm planning on making-because some people just have better eyes for "Wait, did I see a texture seam whipping by back there?" or "I swear these sound effects are cutting out two tenths of a second early", etc. Of course en if they're great at X or Y I'll still assign them W and Z on occasion- being really good at testing just one thing won't really help you career-wise if that's all you're good at.

One of the things I'm thinking about is having something in place that rewards QA as a team for meeting certain goals- a first shot TRC submission, a <50-bug milestone build, etc.- because I was always told that you should reward the group and penalize the individual, praise in public and berate in private, laugh with the door open and yell with it closed, etc. Basically, foster an environment where everyone wants to do well because everyone benefits from it.

Also, anyone have any wisdom for curtailing excessive facebook/reddit usage without going all big brother? It's an issue that need addressing, hopefully without seeming like a tyrant right out the gate.

Doctor Yiff
Jan 2, 2008

Pulling isn't the same as checking out, but whether it's possible is going to depend on what version control you're using.

e: If the dev has a local build or your game works such that they can host a copy of it that the tester can connect to that'll be safer, if you're worried about errantly checking stuff in.

milquetoast child
Jun 27, 2003

literally

GetWellGamers posted:

Eeeehhh, I'm a bit leery of letting bottom-level testers check stuff out of the repository, if only for the same reason I only let young relatives "help" put away plastic cups and dishes- if they drop it, it probably won't break.

I do agree with delegating specific testers to areas they're more interested/proficient in- I mentioned it briefly in the "personality spreadsheet" I'm planning on making-because some people just have better eyes for "Wait, did I see a texture seam whipping by back there?" or "I swear these sound effects are cutting out two tenths of a second early", etc. Of course en if they're great at X or Y I'll still assign them W and Z on occasion- being really good at testing just one thing won't really help you career-wise if that's all you're good at.

One of the things I'm thinking about is having something in place that rewards QA as a team for meeting certain goals- a first shot TRC submission, a <50-bug milestone build, etc.- because I was always told that you should reward the group and penalize the individual, praise in public and berate in private, laugh with the door open and yell with it closed, etc. Basically, foster an environment where everyone wants to do well because everyone benefits from it.

Also, anyone have any wisdom for curtailing excessive facebook/reddit usage without going all big brother? It's an issue that need addressing, hopefully without seeming like a tyrant right out the gate.

Ask them to keep Facebook "to a minimum." It's all right for you guys to check it while pulling down a new build, but don't be alt tabbing out of the game / putting the controller down to chat with your friends all the time. Start off with seeing how it goes with that, then start being more big brothery if things aren't all right.

Just ask them to not keep it open if they're working, but if they're on break go hog wild.

edit: I have an excuse, I'm in social media marketing, so I always have to have facebook open you see. To make sure the channels are still up.

Doctor Yiff
Jan 2, 2008

I wouldn't even bring it up unless it starts to become a problem, but I understand the need to set an expectation. I would think of a method to determine if it's actually excessive use or not - I've met way too many producers and even other testers who couldn't seem to comprehend that QA isn't 100% in the build 24/7 and never takes a break, or couldn't resist brown nosing up a hierarchy that substituted superficialities for actual leadership.

For example, department wide meetings where they'd say it's okay to take short breaks and browse during them, but without fail, would crack a smarmy joke if they saw you on Kotaku, Joystiq or the like, even if you were on a break or waiting for a build. It created an awful, draining, toxic environment where we tried to hide from leadership as much as possible.

Checking in with your team, helping them set goals and advocating for them within the org is going to go a lot farther for their motivation in the long run.

GetWellGamers
Apr 11, 2006

The Get-Well Gamers Foundation: Touching Kids Everywhere!
Well, it already is a problem- it was one of the things they specifically brought up in the interview as something they're looking to me to resolve.

Shalinor
Jun 10, 2002

Can I buy you a rootbeer?

GetWellGamers posted:

Well, it already is a problem- it was one of the things they specifically brought up in the interview as something they're looking to me to resolve.
You may want to observe their behavior before you set a rule on that. That'll tell you if there's actually a problem, or if you're in an environment where management is treating QA like the above stories - which will in turn tell you how you're expected to / should be directing the team.

Juc66
Nov 20, 2005
Lord of The Pants

GetWellGamers posted:

Well, it already is a problem- it was one of the things they specifically brought up in the interview as something they're looking to me to resolve.

Tell them not to do it, and see if you can get IT to see if/when they're opening those pages and have an email automatically sent to you when that happens.

If it's bad enough, see if you can get the sites blocked, or maybe just screw with their hosts file so they get sent to a local network page that says "get back to work" when they try to open up reddit

Feral Bueller
Apr 23, 2004

Fun is important.
Nap Ghost
As a recovering QA manager, both games and non-games:

Try and get your QA people as far upstream in the development process as possible. Are there user stories that can be quickly and easily converted into test cases and acceptance criteria? If not, that's a good place for a lead/aspiring lead/aspiring Product Manager to start.

If Build N+x introduces a defect that wasn't in Build N, that's generally reason enough to fail the build. Doubly so if it reintroduces a formerly resolved defect. This can be controversial in some shops, but I've found it's a good rule to work off of.

Finally, adequate coverage is a group effort. Automated unit testing is awesome if you can get developers to write unit tests for their code.

Doctor Yiff
Jan 2, 2008

Seconding the unit tests. Developers writing unit tests saves a lot of time over the course of the project.

At one point, we had a script that would step through all the game's data files and run basic validation checks. Saved a huge amount of time we would have otherwise blown on discovery, fixing and kicking new builds.

Canned Bovines
Jan 15, 2008

Does anyone have advice for a new technical artist? I spent my internship mostly doing tools development, but I expect I'll be doing a lot more direct support for the artists when I start full time this summer.

Feral Bueller
Apr 23, 2004

Fun is important.
Nap Ghost
Also, a couple of points about mobile testing:

Whether or not the app requires a network connection, be sure to test it in all states:

- WiFi and Cell both ON
- WiFi and Cell both OFF
- WiFi ON; Cell OFF
- WiFi OFF; Cell ON

If you can test on public transportation to check performance as you're passed from one tower to another, that's helpful. I've also tested in the passenger seat with a developer driving. iOS developers seem to be better drivers than Android/Blackberry developers.

Testing with incoming calls:

- Call accepted. What's the application state when user hangs up?
- Call not accepted. What's the application state when the phone stops ringing?
- Call not accepted. What's the application state after the user rejects the call?

Testing with Notifications: see "Testing with incoming calls".

All Operating Systems are not created equal, especially for testing. iOS limits the mix as far as backward compatibility, but carriers matter: you may be able to use your app while on the phone with AT&T, but you might not be able to with VZW.

Android is a backward compatability nightmare, compounded by myriad hardware vendors and multiplied by the greater number of carriers. Make sure you get firm minimum requirements from Product Management. If they say something zany like EVERYTHING, try and get some usage statistics/market share data and try to hit 80% market coverage: you can get a lot of coverage in NA with Samsung, LG, and Amazon Kindle Fire.

Windows Mobile is a very different interface experience: the two projects I worked on started out as "direct" ports and morphed quickly. If this happens to you, make sure the UX deltas are captured and signed off on or you will experience lots and lots of unnecessary pain.

Blackberry = poo poo show.

uTest is a great tool if you can get budget for it. Perfecto Mobile is good if you need to test on a specific device, but can get pricey unless it's managed.

miscellaneous14
Mar 27, 2010

neat

Juc66 posted:

Tell them not to do it, and see if you can get IT to see if/when they're opening those pages and have an email automatically sent to you when that happens.

If it's bad enough, see if you can get the sites blocked, or maybe just screw with their hosts file so they get sent to a local network page that says "get back to work" when they try to open up reddit

This is a bad idea. Occasional browsing never caused issues at the companies I worked for, it's just problem if there's blatantly no work being done.

Sure you could compare it to the actual dev teams who have real goals to reach that require them not to be goofing off occasionally, but they're being paid an actual livable salary and not chump change.

GetWellGamers
Apr 11, 2006

The Get-Well Gamers Foundation: Touching Kids Everywhere!
We're not going to blacklist the sites- my bosses say "Hey, even we like to go to facebook sometimes"- it's just about keeping productivity up enough it doesn't feel like we're paying them to goof off. I mean, yeah, everyone checks their bookmarks a few times during the day, but if you're sneaking games of LoL while you're supposed to be working, that's where I think there's no argument in drawing a line.

Obviously the first step is to just say "Okay guys, cut back on the blogging when you're not on break", but that's been tried already with little success.

milquetoast child
Jun 27, 2003

literally
Haha, I was there for that once.

The network guy installed a trial version of the corporate version of "netnanny" or whatever it is that only had reporting turned on, and was like "boom, you guys spent 4 hours and 20 minutes a day with facebook/myspace/gmail/espn/cnn open as the active window instead of working.

No "action" was taken, just a big demonstration that they knew people were not working so hard, and that they can tell, and that if they don't shape up things might not be so great.

Tricky Ed
Aug 18, 2010

It is important to avoid confusion. This is the one that's okay to lick.



GetWellGamers posted:

We're not going to blacklist the sites- my bosses say "Hey, even we like to go to facebook sometimes"- it's just about keeping productivity up enough it doesn't feel like we're paying them to goof off. I mean, yeah, everyone checks their bookmarks a few times during the day, but if you're sneaking games of LoL while you're supposed to be working, that's where I think there's no argument in drawing a line.

Obviously the first step is to just say "Okay guys, cut back on the blogging when you're not on break", but that's been tried already with little success.

The next step might be to institute some "Browsers closed" hours. Give them the first hour of the day, a half hour on each side of lunch, and 2 hours at the end of the day (obviously breaks should be unsupervised) when it's fine to have external sites open, and the rest of the day is browsers closed. Don't be onerous with penalties or anything, maybe a sin jar if there's any actual penalty, but just communicate clearly that they can actually survive without checking Facebook for an hour. You'll have to hold yourself to this schedule, too.

When I started in QA there was a strict "no external sites except during breaks" policy, and while it grated on me for the first few weeks I got used to it and it kind of helped me kick a bad habit I didn't realize I had. I'll admit I was aiming higher than QA and was on management's side with the policy, though. It wasn't so well received with most of the rest of the team. Then again, those of us who stuck to the policy are all in better jobs now, so...

As for more general stuff, try to develop standard test plans for as much as you can. The more your team can know what to do on their own when they get a new build, the better. You definitely need to schedule ad hoc time, too, but when I was testing an unreleased game my team kind of started rallying around running our standard build tests and getting the results back to the dev team in a shorter amount of time each build.

We'd first split up and do a basic functionality test - about 10-15 minutes per tester, each on a different system, to confirm that we probably had a workable build. If that passed we'd go to our standard tests, where each of us would grab a system and go end-to-end on it as much as possible. After that we'd regress fixed bugs, enter new ones, then start our ad hoc testing. It was a very cool thing where we could see the overall progress in Excel (yeah, actual tracking software is better, but Excel works too) and had a natural inclination to want to do better each time. This is the cool thing about managing gamers.

As it turns out I Have Opinions About QA from being on both sides. I can probably :words: on this for a while so I'll take a bit to consolidate my thoughts.

GetWellGamers
Apr 11, 2006

The Get-Well Gamers Foundation: Touching Kids Everywhere!
Yeah, test plans and case are super vital. When I was at blizzard, they had reams and reams of checklists to cover just about every variable, and when a checklist came back with even a single bad mark, you had to re-do the whole thing the next build just to make sure that in fixing part C of the checklist they didn't accidentally break part A or E. I actually made up some checklists for a few of the places I worked, just to help everyone get done quicker and to avoid duplicating work.

And I guess for me the main difference in internet usage is between "checking" and "browsing". If you need to check your e-mial for 30 seconds every 5-10 minutes because you're waiting for an important message, with maybe a to minute response three or four times a day, I can understand that. Hell, I did that all the time at places that allowed it, juggling between GWG and my "real " job. It's when you go and sit and start reading through threads and articles and browsing feeds I start tapping my imaginary wristwatch.

Also, feel free to go on at length. This is my first professional supervisory position*, so any data or input at all is more than welcome.

Oh, and speaking of Netnanny, there's also apparently widespread rounding-up on breaks- hour lunches lasting an hour and a half, fifteen minute breaks lasting 25 minutes, etc.- so I'll be getting readouts of the punchclock, too.


(*I mean, aside from running GWG, but even then that's managing unpaid volunteers, so it's a very different dynamic.)

Monster w21 Faces
May 11, 2006

"What the fuck is that?"
"What the fuck is this?!"

GetWellGamers posted:

Anyone got any tips for running a QA department? :shobon:

Basic hygiene standards & nip misogyny/racism in the bud.

Leif.
Mar 27, 2005

Son of the Defender
Formerly Diplomaticus/SWATJester
I dealt with a similar issue when I was on my law school's faculty tech committee and we were discussing implementing monitoring programs to prevent students from browsing facebook during lectures. I've also dealt with lazy employees in a supervisory role in my current job and in previous jobs.

It's been said already, but make sure to keep a careful balance in how hard you go after the whole "we're paying attention to how much time you spend on what activity" line. It's inherently poisonous, nobody likes to be monitored. The single most important consideration is how much effective work is being produced. Not just work, but effective work. Work that moves the project towards completion is effective work; work that does not move the project towards completion is not effective work. Keep in mind also that this is not necessarily going to be 8 hours per day, either.

Also the whole checking your email waiting for important message is a terrible habit that should be broken. If the message is so urgent that you need to check on it every few minutes, you are better off calling or walking over (if it is inter-office; out-of-office, for QA, very likely can wait). If the "cost" (in terms of time/energy) of doing that is greater than the benefit of you knowing the information in that message, it's not really as important as you think.

Know which of your employees you can trust with time management. If they can't be trusted to manage their time properly in QA, especially with a known problem of excessive browsing, how can you trust them to manage their schedule properly in Dev (if they are aiming to move up)?

As a supervisor, your job is not to win the love and affection of your subordinates. It's to ensure that they are performing to standard. If that means you need to show some tough love on things like going over alloted break time, that's part of the job. You're their boss, not their friend. You can have friendly relationships; you're an outgoing guy and you're easy to get along with so that's not a problem. But you need to be prepared to be fair, even if that means temporarily pissing off some of the staff because they don't get their way anymore.

One tactic in software (esp. games) is to remind them that every bit of salary is coming out of the budget for the game. When you (regularly) slack off at work, you're harming the game, and you're harming every other employee in the studio by being a money drain in a sector that operates on VERY thin margins.

Of course, if an employee is truly not performing and not responding to your efforts to remedy the problem, that's a signal that maybe you shouldn't keep that worker on anymore.

NetNanny ought to be a last resort -- the poisonous atmosphere of "being watched unfairly" tends to make the "good" workers disgruntled, while the slackers either don't care enough that you're watching, or will find other ways around it.

GetWellGamers
Apr 11, 2006

The Get-Well Gamers Foundation: Touching Kids Everywhere!
I have no problems with using the stick, but I usually resort to the carrot first. And I also don't have problems cracking whips; the groups I've worked for/with in a hobbyist capacity have actually remarked on my ability to motivate them to really put their heads down and work even without any kind of salary or enforceable deadline.

I thin most of all, it's important that subordinates know you care. At least from the other side of the fence, "If they don't give a drat, why should I?" is a surefire productivity killer. At one job I had a boss who basically said "This company sucks and it's never going to change" and it jut hoovered my motivation out of me with how jaded and uncaring they were. Thankfully, their boss was very passionate about our work and maintained that the only way for the company to shed its bad reputation was for the employees, all of them, to make it a better company from the inside out. I wound up talking with them a lot more than my immediate supervisor, because their enthusiasm was infectious.

A Sloth
Aug 4, 2010
EVERY TIME I POST I AM REQUIRED TO DISCLOSE THAT I AM A SHITHEAD.

ASK ME MY EXPERT OPINION ON GENDER BASED INSULTS & "ENGLISH ETHNIC GROUPS".


:banme:
My boss from before blocked Facebook in the hosts file an interns computer because he literally was always chatting to people or laughing at some dumb picture instead of doing work. And one day when there was nobody in but me and the other intern he sat about doing no work, playing a game and then left early. :argh:

A Sloth fucked around with this message at 09:19 on Apr 23, 2013

Sion
Oct 16, 2004

"I'm the boss of space. That's plenty."

GetWellGamers posted:

Anyone got any tips for running a QA department? :shobon:

I did this for about a year up until recently. I may start to ramble. Hopefully some of it is useful.

- Scripted testing is brilliant but it's not a replacement for freeform/destructive. The first reason that it's so great is that it gets people to start thinking in terms of testing a product - it's the sorbet that refreshes the pallet, the five minute jog before a proper work out and the first whiskey of the morning that really sets you up for the rest of the day. A well written test script should fire the imagination of the tester and make them think 'In half an hours time I will come back here and try doing this bit again.'
A test script should really only be about 'does the game fall over if I do X' rather than 'can I make the game fall over if I do Y.' It's an important difference and one that gets forgotten about. Scripted testing isn't what you do when you want to break something, scripted testing is what you do when you want to make sure something works.
The second thing that makes scripted testing brilliant is that it coughs up a success rate percentage. Producers loving love percentages. It makes them feel science-y and like they've for once in their lives gotten actual, firm data to use rather than bullshit sketchy values that they've had to pull out of some guys research paper from a few years ago. If you can break this down by feature (ie. the character customization UI, the sniper rifle's audio, NPC interaction with the player while the player is in kitten form.) you can track how each feature has improved over the course of time by looking at this percentage and going 'pass rate is going up which means it's either getting better or there's some really loving insidious bug that's going to poo poo on our weekends coming up out of it. We should test this area more.'

- Don't freak out. This is something that I have seen a couple of QA managers do (and something I'm guilty of myself) - When you're 12 hours from your publisher submission and someone finds a memory leak in the main menu, take a moment to think how brilliant it is that you found it now rather than before. Then devote some of your team to making sure this issue gets regressed on every possible environment. Take your time, find the positives in the situation (it's great we find out, not so great it made it this far, how can we improve our workflow to make sure it doesn't get this far again) so when the producer sails into the room and asks to speak with you, you're well armed to give them the answers they need/want to hear. Similar to this: Prepare your mind to understand why every single weird call made by otherwise sound QA managers you've worked with in the past was made. The things that made you think 'What the gently caress? Why am I even doing this?' in the past suddenly make you think 'oh. Right. Now I get it.'

- Take a notepad into every meeting. You should do this all the time anyway but when you're in with production or design or programming you'll want to take notes on pretty much everything that gets said. You can use these notes to make scripts out of. Notes are great but for godsake use the confidential shredder if you have one once you're finished with the notebook (do not use those metal ring bound pads if you're going to shred things. I learned this the hard way.)

- If you're asking your cohort to come in and work weekends then please show up yourself, even if it's for a little bit. Make sure that at least someone knows that you're all in-this-together. A fractured department is a deeply unhealthy place to work and quickly becomes a lovely environment for everyone. Showing up, asking if you can get people some drinks or taking folks out to lunch and expenseing it is just fine too and will do great things for morale when you're looking at the 90 hour weeks.
On this note: Sometimes things will not go as planned. Try not to promise anything to your testers because, chances are, you'll have to break that promise due to some issue somewhere else in the company.

True stories of QA time: Day after release on an MMO had been promised as a day off to everyone. During the meeting where this happened someone from QA had specifically asked 'does this include QA?' to whit the answer was a very begrudging 'yes.' Everyone goes to the party, everyone gets tired and emotional and everyone gets an email saying 'Server crash on player enter. P1.' I got called in by the manager who was already in work and asked if I didn't mind printing out the QA phone numbers list and phoning everyone up to bring them into work. I later realized that this was because the manager didn't want to be 'that guy' that ruined everyone's day off. It was the first time I've ever had to prioritize a phone call list by "who has children that they will have promised to spend the day with because I know daddy's been very busy lately..." :smith:

- Sooner or later someone is going to ask you if they can get a raise. They're probably a great tester and one of those ones that somehow manages to maintain a good personal life as well as the 11 hour bastard days that've been dumped on him. The fact that they have a good personal life means they might have a kid on the way. You need to be able to work out a good way of saying no to this guy. Good luck with that.

- Eventually someone's going to put in a bug that fucks someone off upstairs. Maybe they critiqued art, maybe they phrased it poorly or maybe whoever it is in development that got the bug is just having a bad day. Try to be objective here. Your job when this happens is to make sure the poo poo doesn't roll too far down hill but also to make sure that it doesn't happen again. Processes are great here because you're able to avoid a bunch of this stuff. Here's a good process that I've found works, usually even with the most prickly of developers: Write example bugs and send them to everyone ('Brief description like a newspaper headline, brief summary like the blurb on the back of a book and then reprosteps. Attach logs and dumps if available. Submit.') in the department. If they're brand new to QA or the company keep their bugs in a sandbox project on Bugzilla or JIRA - At the end of the day go through those bugs, find what's not right about them and talk to the tester. After a few weeks you should be laughin'.


Monster w21 Faces posted:

Basic hygiene standards & nip misogyny/racism in the bud.

This guy's talkin' sense here. I'm guilty of saying some stupid poo poo in the QA room. Hopefully less in the last couple of years than in the times before then but here's the difficult truth of the matter: For a lot of the people that you work with this is either their first job working with other people in their own age group (and thus, don't understand that there's a difference between hanging out with people of your own age group down the pub and hanging out with a group of people your own age in work) or it's their first job in the games industry.

Basic hygiene standards is also something to watch for but this can be said for any group, really. The thing with QA is that you all tend to get stuffed into one room - the largest local QA team I ever worked with was about 75 people. We had 35 people on one morning shift and 30 people on the evening shift. This meant that there were at least 30 people in one room for 18 hours a day. This didn't leave time for the janitors to come in and do their thing as the only time the room was clear was between midnight and 6 AM. There's very little you can do about things like this. Air fresheners, anti-bacterial hand cleaner and an open window will do good work. If you need to take someone aside and say 'isn't that the same shirt you've been wearing for three days now?' then by all means go for it.

Sion fucked around with this message at 11:30 on Apr 23, 2013

Bruegels Fuckbooks
Sep 14, 2004

Now, listen - I know the two of you are very different from each other in a lot of ways, but you have to understand that as far as Grandpa's concerned, you're both pieces of shit! Yeah. I can prove it mathematically.

Sion posted:

I did this for about a year up until recently. I may start to ramble. Hopefully some of it is useful.

- Scripted testing is brilliant but it's not a replacement for freeform/destructive. The first reason that it's so great is that it gets people to start thinking in terms of testing a product - it's the sorbet that refreshes the pallet, the five minute jog before a proper work out and the first whiskey of the morning that really sets you up for the rest of the day. A well written test script should fire the imagination of the tester and make them think 'In half an hours time I will come back here and try doing this bit again.'
A test script should really only be about 'does the game fall over if I do X' rather than 'can I make the game fall over if I do Y.' It's an important difference and one that gets forgotten about. Scripted testing isn't what you do when you want to break something, scripted testing is what you do when you want to make sure something works.
The second thing that makes scripted testing brilliant is that it coughs up a success rate percentage. Producers loving love percentages. It makes them feel science-y and like they've for once in their lives gotten actual, firm data to use rather than bullshit sketchy values that they've had to pull out of some guys research paper from a few years ago. If you can break this down by feature (ie. the character customization UI, the sniper rifle's audio, NPC interaction with the player while the player is in kitten form.) you can track how each feature has improved over the course of time by looking at this percentage and going 'pass rate is going up which means it's either getting better or there's some really loving insidious bug that's going to poo poo on our weekends coming up out of it. We should test this area more.'

One of the big problems with scripted testing is that it gets over-emphasized by most companies doing QA precisely because it's a process that spits out metrics like "pass %." These metrics tell you nothing about what the tests cover, how long the test cases are, whether or not the test cases are finding bugs, whether or not the test cases are complete, or even whether or not people are actually running the tests correctly (or even understand the tests, assuming they're any good.)

GeeCee
Dec 16, 2004

:scotland::glomp:

"You're going to be...amazing."

Monster w21 Faces posted:

Basic hygiene standards & nip misogyny/racism in the bud.

Counting my blessings that I've never really had to deal with any of this. I'm becoming a little militant over smelly people.

Monster w21 Faces
May 11, 2006

"What the fuck is that?"
"What the fuck is this?!"

Aliginge posted:

Counting my blessings that I've never really had to deal with any of this. I'm becoming a little militant over smelly people.

A lot of the time it isn't so much the people as it is their complete inability to do their laundry properly because they no longer live with Mummy.

Protip, if you don't use a tumble dryer and let your clothes drip dry you'll smell of piss all day, every day.

Leif.
Mar 27, 2005

Son of the Defender
Formerly Diplomaticus/SWATJester
If you bathe in piss, maybe. If you use detergent like a normal person, drip drying shouldn't smell like piss, or anything else.

The sad thing is that after two years of living in a third world developing country in Africa (where most people bathe weekly at best), I can conclusively say that I have been to FLGS and conventions that smell decidedly worse.

-e- the point about laundry is spot on though.

Doctor Yiff
Jan 2, 2008

If there's one thing I don't miss about games QA it's the default assumption that I'm going to smell like piss, require monitoring software on my pc or need rules about 'browser hours' in order to actually do my job.

FreakyZoid
Nov 28, 2002

Sion posted:

- Eventually someone's going to put in a bug that fucks someone off upstairs.
It's funny reading this, because having never worked in QA but having ran design teams, my only experience is of talking to developers who'd written terrible responses to bugs and tried to have them closed or moved off their plates. The rule of thumb I always preached was "don't be a dick on (bugtracking software)". Along with "Imagine how you'd feel if you were having a terrible day, maybe your dog died or your kid hadn't slept all night, and you turned up to work to see the response you'd just written. How would you feel? Now, do you want to change what you've written?"

Monster w21 Faces
May 11, 2006

"What the fuck is that?"
"What the fuck is this?!"

FreakyZoid posted:

It's funny reading this, because having never worked in QA but having ran design teams, my only experience is of talking to developers who'd written terrible responses to bugs and tried to have them closed or moved off their plates. The rule of thumb I always preached was "don't be a dick on (bugtracking software)". Along with "Imagine how you'd feel if you were having a terrible day, maybe your dog died or your kid hadn't slept all night, and you turned up to work to see the response you'd just written. How would you feel? Now, do you want to change what you've written?"

Welcome to the games industry, where the libertarian mind set runs free and apathy is a luxury trait.

Jan
Feb 27, 2008

The disruptive powers of excessive national fecundity may have played a greater part in bursting the bonds of convention than either the power of ideas or the errors of autocracy.
Where apathy is a necessity.

devilmouse
Mar 26, 2004

It's just like real life.
My two favorite bug responses that I've seen over the years (I've probably posted them before, but I can't not when the opportunity presents itself!).

The first is in response to a bug that said only "What are these?" with an attached screenshot.

quote:

11/17/2005

the correct summary for this bug is not "what are these?", but "what is this?". and the correct answer to that question is "this is the most poorly submitted bug of all time, and I should not even loving acknowledge its existence until it reappears properly formatted in his bug queue". as the most unprofessional and rude member of the dev team, it's impressive that even i am a little indignant. this is just loving sloppy, cupcake. rookie poo poo. never do it again. that said, this is an easy fix. i just changed the particles and their material to be lit, and we're good to go. they now light appropriately with the scene and look fine against the dark sky. of course, based on the summary given, i had no idea what kind of task i was getting into. this is a problem for me when looking at my bug queue, yeah? which is not good for any of us. do not think for a minute i am fixing this because i consider it a high priority task, i am fixing so i can write this rude reply and be done with this stupid, stupid bug.

The second should be self-explanatory:

quote:

===== Opened by ******* on 07/11/2002 03:02PM =====
Human Male Avatar has visibly longer left arm in AVATAR CREATION's gender/race selection screen.

Repro Steps:
1. From starter screen, Click Create.
2. When "Choose Your Avatar" gender/race screen comes up, observe the left arm (the one closest to the wall) on the human male.

Arm seems slightly bigger and longer than the arm that is closest to the viewer. It is more apparent in the idle animation when char looks towards the wall, and during the mouse over animation.

===== Resolved as Won't Fix by ******* in 00.00.00.0000 on 07/25/2002 10:40AM =====
That's because the right arm is pointed towards the camera - and when you take two objects of equal length, and one is pointed AT the camera, it looks smaller because you're not viewing it long-ways anymore. This is also known as "Foreshortening", in art school. The two arms are exactly the same in the model. This effect is part of what makes the game "Threeeeee-Deeee".

Monster w21 Faces
May 11, 2006

"What the fuck is that?"
"What the fuck is this?!"

Jan posted:

Where apathy is a necessity.

Look at me using the wrong word. Empathy.

D1Sergo
May 5, 2006

Be sure to take a 15-minute break every hour.
edit: I read it wrong.

D1Sergo fucked around with this message at 16:19 on Apr 23, 2013

devilmouse
Mar 26, 2004

It's just like real life.
Edit: nm you read it wrong! =)

Doctor Yiff
Jan 2, 2008

Nah, dev is right on the facts in this one. Left arm looks longer because the camera is perpendicular to the arm instead of parallel.

I would have opened up the model myself to check my assumptions (assuming I even could, which is why I think it's important for QA to be able to access game data). If I'm being charitable, I'd say tester and dev were under a lot of strain, but I have some context about devilmouse's old haunts.

Comte de Saint-Germain
Mar 26, 2001

Snouk but and snouk ben,
I find the smell of an earthly man,
Be he living, or be he dead,
His heart this night shall kitchen my bread.

Monster w21 Faces posted:

A lot of the time it isn't so much the people as it is their complete inability to do their laundry properly because they no longer live with Mummy.

Protip, if you don't use a tumble dryer and let your clothes drip dry you'll smell of piss all day, every day.

No one in europe has tumble dryers and the only person at this company who smells like piss is an american. Maybe you are just doing it wrong.

Senso
Nov 4, 2005

Always working

Comte de Saint-Germain posted:

No one in europe has tumble dryers

In laundromats they have. :colbert: It's better than bringing back home a bunch of soaked clothes.

Shalinor
Jun 10, 2002

Can I buy you a rootbeer?

Comte de Saint-Germain posted:

No one in europe has tumble dryers
This seems downright alien - do you guys all seriously have clotheslines on your back porches? How does that even work if it's, say, snowing outside (like it has been here for the last week)?

GeeCee
Dec 16, 2004

:scotland::glomp:

"You're going to be...amazing."
I have a tumble dryer :buddy:

EDIT: While I'm bragging I have a beautiful view over all dundee from my flat too. :D

Adbot
ADBOT LOVES YOU

mutata
Mar 1, 2003

Shalinor posted:

This seems downright alien - do you guys all seriously have clotheslines on your back porches? How does that even work if it's, say, snowing outside (like it has been here for the last week)?

In urban Ukraine, everyone line dried their clothes on their balconies. Dryers exist, they just take up space that we didn't have.

Seriously, if you look at the labels of your clothes even here in America, half of them say to hang-to-dry, even more for women's clothes. Whoever said that hanging clothes to dry makes them smell like piss is weird.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply