undefined | Better HN

0 pointsmikestew8y ago0 comments

Take this for the anecdata that it is. I interviewed at Apple, referred by old Microsoft friends that worked there. As I was trying to get a feel for things before the interview, I asked about the software testing. I was told, "don't expect what you're used to at Microsoft". The reference there is from when Microsoft often had more testers on a team than devs (ah, the good ol' days). The summary of what I was told by friends, and the questions I asked during the interview, is that testers at Apple aren't the testers that Microsoft used to have. Microsoft had testers working in MS Research, researching ways to better test software. Apple, from the impressions I got, is doing good to have testers than can write "hello, world". This was from the app side of things, not OS; I don't know if it's any different on the OS side.

But since I don't work there, I have no good inside info. But just from gut feel, I don't think my anecdata is too far off the mark. Based just on the bugs made public, I just don't get the impression that there are testers at Apple whose sole reason for being there is to tear into a piece of software and break it. There was a bug a few weeks ago posted to HN that I commented on. I don't have a link without digging through my comments, but it was something along the lines of "how could a tester not find this in five minutes of exploratory testing?" This bug is similar. It would take more than five minutes, but were this my area to test I'd pick at it once in a while when I had a few minutes. As I pick at it, I wouldn't expect to find anything, but I've got a minute between builds, so instead of randomly clicking Facebook I'll randomly click this dialog. What did the dev forget? What weird state was not accounted for? Some kind of state overflow if I click the button enough times? Shove some Unicode in there, that didn't find anything; meh, maybe I ought to move o...hey, wait a minute. Did that thing just log me in as root?

But my gut says that Apple doesn't employ a lot of testers like that.

0 comments

jenoer8y ago

As a Tester myself, I cannot understand why this is not covered by either unit tests or behavioral tests. Clicking dialog buttons in rapid succession is what we (should) do once in a while. Especially in core functionalities such as the login screen. It's one of the first screens you see as a tester. And you have default usernames, be it enabled or not.

For example, I do not own an iPhone, but at work, I made a bet with my colleague (jokingly) that I could break _something_ on his phone in a few minutes.

I did not have his finger print or pin-code, so I was very limited, I even joked "I don't need that, give it here!"

Finding out I only had a hand full of options, I focused on the emergency dialer. As any good tester would be curious about, I wanted to check the max field length, so I entered digits, copy/paste it a few times, copy/paste that string, ("wait, no limit? Not even at 1000? why?") and so on, until I noticed the interface became laggy, so of course, I kept going.

Boom, suddenly back at the login screen, tried to open the emergency dialer, but got a full blank white screen, in the meantime the phone started heating up substantially. Since it was a new Phone (iPhone 7 with iOS 10.x I believe) and the dev getting nervous, we decided to reboot it. That fixed the issue. (Curious if this is still an issue in iOS 11.x)

TL;DR: As a tester this simple curiosity should be in your blood, and especially covered in behavioral tests when your software has been around for 5+ years.

MrMid8y ago

I got my friends Apple Watch stuck last week just by looking at it's features. IIRC it got stuck while using the "flashlight". It suddenly froze, and it took me a while to reboot it (it got stuck once more while rebooting).

All in all, it took me about a minute to break it, and around 5 minutes to get it working again. I was getting a bit nervous.

Cthulhu_8y ago

This bug isn't caused by rapid succession or whatever, it's more of a generic end to end test. In this case someone would have had to write an exact scenario that opens a settings page, unlocks it, types in 'root', no password, presses login and it should not work.

jenoer8y ago

Rapid succession button clicking usually combines many different tests:

- Performance

- Usability

- Brute-force capabilities

- Error handling

- And in this case, Security, a bug where trying to log in a couple of times on the Login screen with an empty, or set, password.

A test scenario closer to this would be:

---

When I am on the login screen

And I enter 'root' in the 'Username' field

And I enter 'thispasswordisfalse' in the 'Password' field

And I press the 'Login' button '10' times

Then I should see the text 'Your password is invalid'

---

Please note that this issue is not just in the Settings page, it takes place on the login screen as well, that's why I'm shocked, it's such a core functionality, touching so many system components.

tyingq8y ago

In this case, any password works. The blank one is just the initial example. The password is set to whatever you typed, even if nothing.

yodsanklai8y ago

> But since I don't work there, I have no good inside info

Actually, I've been wondering why I hear less about people working at Apple than at other big tech companies. It seems everyone and their mother work at Google or Facebook, but no so much at Apple. Do they have less software engineers, or their employees are required to be more discrete?

mikestewOP8y ago

Do they have less software engineers, or their employees are required to be more discrete?

I know but a few that work at Apple, and of those few they strike me as less forthcoming than the multitudes I've worked with and know at Microsoft. I've wondered if part of that is because Microsoft previews/pre-announces just about everything, whereas Apple (mostly, and not so much anymore) announces it when the shipping trucks show up at the local Apple store.

So the outcome from the Microsoftie is, "it'll do this that and the other, but that's all I can say right now." From a recent conversation with an Apple employee: "they make me go in a special room to use the hardware, and I can't work from home. That's all I can say."

Probably more so, last I looked, Apple has considerably fewer software employees than the other big companies.

saagarjha8y ago

> Probably more so, last I looked, Apple has considerably fewer software employees than the other big companies.

I don't think this is true. Apple, Google, and Microsoft all have on the order of 100K employees.

1 more reply

OJFord8y ago

> or their employees are required to be more discrete?

Yes, I believe so. I've heard there are strict requirements on even internal discussion. (Who you can talk to; about what; where.)

addicted8y ago

I believe there was an article about how Apple was finding it extremely difficult to hire ML experts because of the secrecy they require. In order to mitigate that they mad an exception for their ML/AI engineers allowing them to publish papers to external journals and present at conferences.

qhwudbebd8y ago

Presumably this is why their software is increasingly shoddy. But I wonder whether it's a direct effect of poor internal communications, or an indirect effect where the ludicrous secrecy has driven away any half-decent programmers.

wil4218y ago

Could it be the level of secrecy around Apple? I see responses for Google and Facebook devs on HN a lot but never Apple.

The only people I know locally that work for Apple are remote customer support folks.

skc8y ago

Apple probably doesn't take too kindly to their employees talking about their work. I'd imagine it's a fire-able offense.

chris_78y ago

it is everywhere else too, but people do quit...

j / k navigate · click thread line to collapse

0 pointsmikestew8y ago0 comments

But my gut says that Apple doesn't employ a lot of testers like that.

0 comments

jenoer8y ago

For example, I do not own an iPhone, but at work, I made a bet with my colleague (jokingly) that I could break _something_ on his phone in a few minutes.

I did not have his finger print or pin-code, so I was very limited, I even joked "I don't need that, give it here!"

TL;DR: As a tester this simple curiosity should be in your blood, and especially covered in behavioral tests when your software has been around for 5+ years.

MrMid8y ago

All in all, it took me about a minute to break it, and around 5 minutes to get it working again. I was getting a bit nervous.

Cthulhu_8y ago

jenoer8y ago

Rapid succession button clicking usually combines many different tests:

- Performance

- Usability

- Brute-force capabilities

- Error handling

- And in this case, Security, a bug where trying to log in a couple of times on the Login screen with an empty, or set, password.

A test scenario closer to this would be:

---

When I am on the login screen

And I enter 'root' in the 'Username' field

And I enter 'thispasswordisfalse' in the 'Password' field

And I press the 'Login' button '10' times

Then I should see the text 'Your password is invalid'

---

Please note that this issue is not just in the Settings page, it takes place on the login screen as well, that's why I'm shocked, it's such a core functionality, touching so many system components.

tyingq8y ago

In this case, any password works. The blank one is just the initial example. The password is set to whatever you typed, even if nothing.

yodsanklai8y ago

> But since I don't work there, I have no good inside info

mikestewOP8y ago

Do they have less software engineers, or their employees are required to be more discrete?

Probably more so, last I looked, Apple has considerably fewer software employees than the other big companies.

saagarjha8y ago

> Probably more so, last I looked, Apple has considerably fewer software employees than the other big companies.

I don't think this is true. Apple, Google, and Microsoft all have on the order of 100K employees.

1 more reply

OJFord8y ago

> or their employees are required to be more discrete?

Yes, I believe so. I've heard there are strict requirements on even internal discussion. (Who you can talk to; about what; where.)

addicted8y ago

qhwudbebd8y ago

wil4218y ago

Could it be the level of secrecy around Apple? I see responses for Google and Facebook devs on HN a lot but never Apple.

The only people I know locally that work for Apple are remote customer support folks.

skc8y ago

Apple probably doesn't take too kindly to their employees talking about their work. I'd imagine it's a fire-able offense.

chris_78y ago

it is everywhere else too, but people do quit...

j / k navigate · click thread line to collapse