App Validation fails only on Kindle Fire HD 8 2020

You are right. When you submit your zip or index, Amazon is running Cordova against it to produce the apk. You should get a very similar apk, when you use Web App Tester (it does Cordova internally).

Potential problems are coming from the fact that Cordova version that Amazon is using is rather prehistoric - v3, thus from time to time there are problems with the latest FOS. Apparently, Amazon lost people that were doing Cordova work, or doesn't want to pay consultants who delivered the code in the first place.

With all that, most likely that should not be the fault with your code, but rather Amazon butchering the build, test group using the wrong apk.

Saying that, we have also submitted web apps over the past few days. Those submitted earlier got stuck, due to broken submission framework. However, today we got one app approved for both HD 10 2019 and HD 8 2020 (one that your app failed). Although, looking at our server logs, India didn't spend too much time testing this time.

Therefore, there is a possibility that when your HD 8 build was running, Amazon screwed something up without realizing it. Or, maybe they looked at your post, and fixed the issue, so we got lucky.

If you are still stuck, just resubmit your app as is, and hope for the best.

Thanks, Jacek

PS.
I assume that you also selected HD 10 2019, and that one passed. Both HD 10 2019 and HD 8 2020 are FOS 7, and have the same Android and API level. Therefore, it is rather not possible that one device will pass and the other will fail, as they run from the same apk and they share the same chromium level.

I would also add to 'Testing instructions' that since HD 10 2019 passed, there is no reason for HD8 to fail, plus that is not you who is doing the build as that is a web app. Also, I would also do Contact Us, letting them know that they butchered the process. That is just CYA on your part, if Amazon wants to slow you down.

@Mike@Amazon

I think you misunderstood the issue. It is not about the app being live on HD 8 2020, but rather the app not always starting properly on HD 8 2020, HD 10 2019, and all Fire TV devices. This issue was not there on Fire TV devices before Nov 1st FOS update.

The problem that the submission team saw when originally testing on HD 8 2020 has been propagated to all Fire TV devices that run the latest FOS (with chromium v84). It looks like it is affecting about 10-20% of first downloads.

As at that time, the submission team didn't bother to provide console logs, and we cannot reproduce it in-house (potentially due to a limited number of devices), it would be prudent for Amazon to retest one of our apps, and send us a console log when the app will either not start, or the UI will get mangled (as one end user reported on Amazon info page about our app).

The Contact Us #7451817491 has more info.

Actually, the case is more urgent, as the app mentioned in that Contact Us case is a paid-on-download app, so we both (Amazon and us) are potentially losing revenue and customers.

Thanks, Jacek

Hi,

I'm happy to see that your app is live and available on the Kindle Fire HD 8 2020.

Please let us know if you have any other issues by using the Contact Us form.

Thanks,

Mike

@Levon@Amazon

Looks like, this is a bigger problem. As already noted, Amazon transitioned those apps that got stuck on Fire Tablet 8 2020 to the Live status. However, looks like the original issue is still there, and actually got propagated to the latest FOS with chromium v84 which also include Fire TVs (not just Fire Tablets). What we see is that for some users, apps potentially don't start (server logs are exactly the same as those when Fire HD 8 failed - all logs are good up to the point where the UI should be there), but looks like users abandon those apps (no other activities). We also got Amazon review for such start stating that the app didn't really start, or rather that UI was corrupted.

So, we are back to square one. When Amazon tested and saw those failures, the submission team didn't bother to provide console logs. Also, we were not able to reproduce those failures in-house.

As those failures are right now happening on both Fire Tablets and TVs, it would be helpful, if Amazon could run some tests, and provide console logs when those failures happen, as it should be doing when seeing those problems on Fire HD 8 2020 devices.

Thanks, Jacek

Another submission, another failure on HD 8 2020. Again, no screenshots, no console log.

Actually, the left hand side 'feedback' field is called 'Steps to reproduce.' What are the steps? Are those testers too dumb to understand the difference between 'steps to reproduce' vs. 'general feedback?'

Similar app (exactly the same code) was approved yesterday, and passed on that device.

A couple of weeks ago, Amazon started fixing those failed HD 8 devices without even asking us whether to do it or not. And we are now going back to the square one.

Is it so hard to tell those damn testers to provide screenshots and console logs? Steps to reproduce? Do they not speak any English?

Is it also so hard for them to push back the build to the team that does Cordova wrapping, to have them look at what is screwed up on FOS 7.

What gives? How long this shitshow will continue?

Looks like Amazon figured out what went wrong with testing on HD 8 2020, and is re-enabling (web?) apps that failed. However, it may not be automatic.

We were asked (through Contact Us case) to provide them with ASINs to what is still listed as failed, and hopefully they will re-enable those.

@Levon@Amazon

It is over a week, since this issue was brought up. Any progress, or rather anyone cares?

I did Contact Us case, asking to retest one app that failed that device, and requested a log and screenshots be provided, but so far crickets.

@Levon@Amazon

We may have either HD 8 2020 web app apk build problem, or submission testing problem on that device is inconsistent.

Today one of our apps also failed on HD 8 2020. Although, other apps passed on that devices (also today).

1. There are no logs provided (under 'Functionality Validation / Download' - 'Not available'). Although, looking at the server logs, the app UI fully started. There are also no errors in our server logs pointing to an obvious problem. Whether that UI was visible or not is a secondary issue. However, if the app managed to run up to that point, there should be logs in the console. It could be no logs, if there are usually syntax errors with the code, and the app doesn't manage to start at all. So, not seeing that log there looks not right.
2. There are no screenshot provided (was there a spinner on screen, which one (we have two, indicating the starting process), were there any UI elements present).
3. Looking at server logs, our other apps with exactly the same logs passed (although, we have rather sparse logs, so cannot tell what exactly happened, after the app started).
4. The app passed on HD 10 2019. That is basically the same devices, or very similar. Same FOS, same Android/API level, same chromium version. Similar HW, although CPU/GPU are different.
5. Our app code that passed on that device is exactly the same as the code that failed on that device.
6. We don't do any code specific checks to test for the newest devices, rather check for low chromium version, as some JS/CSS may not run on old web view/chromium.
7. From time to time, we do see in the field people abandoning our apps right after the start also on other devices, but cannot tell whether the app didn't start, or the end users after short browsing decided to kill it. We have also previously reported that we feel that the very first starts on a clean devices may not always work. However, from what we can tell, if the user decides to run the app once more, looks like it always works (i.e., there are some interactions reported to the server).

Unfortunately, at the moment, we don't have in-house HD 8 2020, as we assume that HD 10 2019 is basically equivalent one (looking at FOS and web view).

Thanks, Jacek