(Topic ID: 173145)

WoZ 6.xx attract mode lockups - HARDWARE FIX AVAILABLE


By vireland

2 years ago



Topic Stats

  • 23 posts
  • 8 Pinsiders participating
  • Latest reply 1 year ago by check_switch_26
  • Topic is favorited by 8 Pinsiders

You

Linked Games

Topic Gallery

There have been 6 images uploaded to this topic. (View topic image gallery).

IMG_2989 (resized).JPG
IMG_2981 (resized).JPG
IMG_2983 (resized).JPG
IMG_2972 (resized).JPG
IMG_2973 (resized).JPG
IMG_2974 (resized).JPG

#1 2 years ago

EDIT - FIXED! See post 12 for the fix (If you have an ECLE with a MSI H61M-P31/W8 motherboard and Celeron G540 2.5Ghz CPU, you likely have this problem)

I'm trying to track down the problem on a 2013 ECLE WoZ that was upgraded to 6.06 (then downgraded to 6.04 while trying to solve this) where it locks up in attract mode at least once daily. The playfield animations still happen, but the screen freezes and start button doesn't work. Coin door opening or service menu doesn't break it out of the freeze (unlike Hobbit 1.90, where opening the coin door "unsticks" it from the high score entry lockup).

Reverting to 5.05 solves the problem, so it's most likely a software issue present in Ubuntu 15.10. I've read that asynchronous transfers with an SSD can cause lockups in 15.10, depending on what kernel, but I don't know what kernel WoZ 6.06 is using.

I know I'm not the only one having this problem (verified), but are any other Pinside users seeing this? You pretty much have to leave the machine on all day and night to see one lockup, although some days you might see two. I've been working with JJP to try to nail this down, but knowing even more users with the issue may provide enough datapoints to make it easier to track down the root of the issue to get this fixed.

#2 2 years ago

It's running 4.2.0-16-generic.

Haven't run into the issue myself, but I haven't let the machine run that long either.

#3 2 years ago
Quoted from DeeGor:

It's running 4.2.0-16-generic.
Haven't run into the issue myself, but I haven't let the machine run that long either.

There's a lot of reading on users with computer freezing issues with Ubuntu 15.10 on their PCs, but they seem to be confined to 4.2.xx and up. 4.1.xx range seems to not have the problems. I wonder if a minor downgrade to a more stable/compatible kernel is even possible.

#4 2 years ago

Downgrading the kernel version would be difficult.

It would be nice to get some logs off the machine during the time of the crash / lockup. Does the machine remain locked up indefinitely when this happens or does it eventually resolve itself?

#5 2 years ago

I had this on my location game. Reseated the ram solved the problem. Although I'm not sure which version I was running.

#6 2 years ago
Quoted from DeeGor:

Downgrading the kernel version would be difficult.
It would be nice to get some logs off the machine during the time of the crash / lockup. Does the machine remain locked up indefinitely when this happens or does it eventually resolve itself?

The screen is locked and buttons unresponsive (including the service menu) though the playfield animations still cycle. Stays that way until power cycled.

#7 2 years ago
Quoted from pinballkyle:

I had this on my location game. Reseated the ram solved the problem. Although I'm not sure which version I was running.

I don't know how reseating RAM would help with this if version 5.xx works fine. If it were a RAM issue, it seems like any version would have problems. That said, I will try reseating it.

11 months later
#8 1 year ago

Was there a solution? My day one ECWOZ is doing this now. Locks up in attract mode after a few hrs. I am able to start a game and play one ball. when it drains its over. When the game starts I have no music but lion call out for being scared comes on. During game game play call outs work but no other music. Rebooting cures it but got too figure it out. I have a ticket open. I am running 6.06 code.

#9 1 year ago
Quoted from nitrojcrawf:

Was there a solution? My day one ECWOZ is doing this now. Locks up in attract mode after a few hrs. I am able to start a game and play one ball. when it drains its over. When the game starts I have no music but lion call out for being scared comes on. During game game play call outs work but no other music. Rebooting cures it but got too figure it out. I have a ticket open. I am running 6.06 code.

The only current solution is to revert to 5.05, which works perfectly. All flavors of 6.xx will lock up sometime within 24 hours on machines with the issue.

I really would like trying the newer motherboard from The Hobbit (and newer WoZ) since I think it's basically a Linux chipset driver problem with the newer kernel of 6.xx and up on the existing motherboard from the oldest ECLEs.

#10 1 year ago

Thanks for the heads up. I’ve got a ticket open with them on this. I will post if anything comes of this

#11 1 year ago
Quoted from nitrojcrawf:

Thanks for the heads up. I’ve got a ticket open with them on this. I will post if anything comes of this

If you make any progress getting to the root of the problem and an actual fix, definitely post in this thread. I know quite a few that have been waiting for a solution for a long time.

3 weeks later
#12 1 year ago

So the software fix still hasn't happened. I decided to just replace the early run WoZ motherboard/cpu (MSI H61M-P31/W8 with a Celeron G540 2.5Ghz CPU) with the motherboard in later WoZ and The Hobbit that DO NOT have lockup issues (MSI H81M-P33 and an i3-4170 CPU). After the upgrade, it's on the fourth day without locking up running the latest 6.50 software, and with the old motherboard it would have locked up at least once in the first 24 hours, so I'm calling this fixed. Takes about a hour to swap everything and get it back up. Here's what you need:

H81M-P33 motherboard ($46)
amazon.com link »

LGA1150-compatible i3-4170 processor ($144 - required since the new motherboard requires a Haswell CPU, original CPU won't work):
amazon.com link »

Thermal compound (don't use what comes already on the heatsink with the processor - scrape that off and use this):
amazon.com link »

Take PICTURES of the 6 connectors on the old motherboard and what they connect TO since the positions are different on the new motherboard, but the screened names of each header block are the same. You just need to match them back up using the pictures you take before the swap:
1. The 20 pin power connector (note that the motherboard will take a 24 pin connector, but the last 4 are empty - this is the same on the original and new motherboard)
2. The 4 pin yellow/black aux connector
3. The USB1 connector
4. The audio connector
5. The 2 pin black/red reset/power connector
6. The CPU fan connector

Make sure you connect a keyboard to the USB. Then, note that once these are connected, you will have to hit the reset button the first time to manually turn on the computer. The reset button is near the bottom of the computer case, toward the coin door. It was clearly marked. Once you change the BIOS settings, the motherboard will automatically come on when power is turned on.

I've attached the screens that need to be changed in the new BIOS to match the original WoZ board. Just change the settings as-needed on the new motherboard to match the screens below taken of the old motherboard settings. On the boot order screen where everything's doubled, one is UEFI and one is regular for each of the instances on the new motherboard.

IMG_2974 (resized).JPG
IMG_2973 (resized).JPG
IMG_2972 (resized).JPG
IMG_2983 (resized).JPG
IMG_2981 (resized).JPG
IMG_2989 (resized).JPG

#13 1 year ago
Quoted from vireland:

So the software fix still hasn't happened. I decided to just replace the early run WoZ board with the motherboard in later WoZ and The Hobbit that DO NOT have lockup issues. It's on the third day without locking up running the latest 6.50 software, and with the old motherboard it would have locked up at least once in the first 24 hours, so I'm calling this fixed. Takes about a hour to swap everything and get it back up. Here's what you need:
H81M-P33 motherboard ($46)
amazon.com link »
LGA1150-compatible i3 processor ($144):
amazon.com link »
Thermal compound (don't use what comes already on the heatsink with the processor - scrape that off and use this):
amazon.com link »
Take PICTURES of the 6 connectors on the old motherboard and what they connect TO:
1. The 20 pin power connector (note that the motherboard will take a 24 pin connector, but the last 4 are empty - this is the same on the original and new motherboard)
2. The 4 pin yellow/black aux connector
3. The USB1 connector
4. The audio connector
5. The 2 pin black/red reset/power connector
6. The CPU fan connector
Make sure you connect a keyboard to the USB. Then, note that once these are connected, you will have to hit the reset button the first time to manually turn on the computer. The reset button is near the bottom of the computer case, toward the coin door. It was clearly marked. Once you change the BIOS settings, the motherboard will automatically come on when power is turned on.
I've attached the screens that need to be changed in the new BIOS to match the original WoZ board. On the boot order screen where everything's doubled, one is UEFI and one is regular for each of the instances on the new motherboard.

If this fix remains stable . Are going to pass this info on to Frank @ JJP Tech support ? Good job !!!

#14 1 year ago
Quoted from RobKnapp:

If this fix remains stable . Are going to pass this info on to Frank @ JJP Tech support ? Good job !!!

I already sent this to Frank, Jack, Butch, Keith, etc, after it was stable for the first 24 hours. I waited another day and a bit before posting so I could be sure it was fixed. It's not free like a software fix, but the software's been broken for almost 2 years and I was tired of waiting. I think most OPs would take a $200 for sure fix over a free "maybe" fix sometime in the future, especially after how long it's already been with no fix, and in a number of cases no admission of the problem.

#15 1 year ago
Quoted from vireland:

Thermal compound (don't use what comes already on the heatsink with the processor - scrape that off and use this):
amazon.com link »

You can skip the special thermal compound. You're not running anywhere near the cooling limits of the pad that comes pre-installed with the stock heat sink. It's an additional step for no gain.

#16 1 year ago
Quoted from YeOldPinPlayer:

You can skip the special thermal compound. You're not running anywhere near the cooling limits of the pad that comes pre-installed with the stock heat sink. It's an additional step for no gain.

Easy to do it now since the thermal paste has a much longer life than what comes with the heatsink. I'm a big fan of one and done.

1 week later
#17 1 year ago

This problem is fixed in version 6.60 of the software.

--Ted

#18 1 year ago
Quoted from check_switch_26:

This problem is fixed in version 6.60 of the software.
--Ted

Hopefully this is the real fix. I personally installed multiple versions of different 6.xx that were supposed to be fixed that weren't before I gave up and eventually just changed the hardware.

I hope the next time a Linux driver bug pops up, JJP doesn't leave operators hanging with machines that lock up for two years (WoZ) or 6+ months (Hobbit sound popping) with no solution, not even acknowledging the known issue to some of the Ops.

#19 1 year ago
Quoted from vireland:

Hopefully this is the real fix.

I'm pretty confident this fixes it. I tested it for days longer than you tested your hardware fix before declaring victory. LTG's Oz game freezes 2-3 times a day with 6.50. It has been running for three days problem-free with 6.60.

--Ted

Editing to add: I was extra careful to fully test this before declaring a win. I knew I'd never hear the end of it if you went to the trouble of swapping your old motherboard back in and your screen froze up. What is it with your repetitive negativity anyway? And by the way, this wasn't an easy fix to find OR test, which is why it took so long to resolve the problem. Oh, and you're welcome.

#20 1 year ago

I was having lock ups on 6.06 thats why i didn't update to 6.50 for the longest time. I was having almost one a day since starting to play again now that life has slowed down. On suggestion from JJP tech I went into the control box and reseated all usb/mirco usb and video connections. I also reseated the crystal ball connector. After I did that i had no more lock ups for a week. I updated to 6.50 and still nothing. will be updating to 6.6 tomorrow and hope this weekend is good too.

#21 1 year ago
Quoted from check_switch_26:

Editing to add: I was extra careful to fully test this before declaring a win. I knew I'd never hear the end of it if you went to the trouble of swapping your old motherboard back in and your screen froze up. What is it with your repetitive negativity anyway? And by the way, this wasn't an easy fix to find OR test, which is why it took so long to resolve the problem. Oh, and you're welcome.

Thanks for asking. My negativity stems from the fact that this issue was talked about on the Linux PC boards almost immediately after the kernel came out with people complaining about freezing video on their computers. There were many workarounds listed, with success reported. Those workarounds could have been tried on the affected WoZ machines. In the next two years despite repeated offers for our 100% affected route WoZ to be a guinea pig to try to help JJP find the solution with different workarounds...nothing. Meanwhile, I know personally other OPs that were having the issue and when they approached JJP, they acted like it was the first time they ever heard of the problem - more than a year in. I had to host 5.05 (JJP had taken 5.05 down sometime in early 2017, I think, despite it being the only fix for affected machines) so the Ops I told about the solution could fix their machines.

I don't doubt it wasn't easy to find or test, but I guarantee finding it and testing it wasn't a priority at JJP for all of those two years, and since the motherboard and/or chipset was suspect all along, a simple message of "hey, we don't have a software fix, but for $200 you can swap out the motherboard and CPU that will fix the problem since our newer WoZ with this new motherboard are unaffected" would have saved two YEARS of problems with the machine on location and countless re-installs. I know I would have taken that deal.

I definitely think JJP is getting better with this back-end stuff (and tech staff like Frank are without reservation GREAT to deal with), but this was by far the worst example of poor operator support for a JJP machine. We've supported JJP on every one of their releases so far, and intend to KEEP supporting you guys because on balance you're the best things out there now, but I'm not going to candy-coat opinions about issues affecting operators that are known and run 6-12-24 months long with no solution because it hurts someone's feelings.

Also, note the date on this thread. I didn't start complaining publicly and looking for answers until a YEAR went by with no fix. I wasn't hair-trigger bitching about a pretty major problem for the location we had it at.

It all worked out, so I'm not putting the old motherboard back in. The newer WoZ/Hobbit motherboard is WAY faster to load and supports 1080p, so if I change the monitor at some point I'll be set. The $200 is already spent, the work done, so no sense in reverting just to get the JJP boot screen (some day I may flash the rom to put that in, for now the 2 second MSI screen is no issue).

#22 1 year ago

First of all Ted thank you my only complaint is I asked about this issue and was not told anything it's the not knowing if it will ever be fixed that was concerning but wow just wow thank you for fixing it.

#23 1 year ago
Quoted from vireland:

[T]his issue was talked about on the Linux PC boards almost immediately after the kernel came out with people complaining about freezing video on their computers. There were many workarounds listed, with success reported. Those workarounds could have been tried on the affected WoZ machines. In the next two years despite repeated offers for our 100% affected route WoZ to be a guinea pig to try to help JJP find the solution with different workarounds...nothing.

And we did try all of those workarounds. We can Google just as well as you can. The fix was not found online, and in fact some documentation out there states the opposite of what eventually led me to the solution.

I guarantee finding it and testing it wasn't a priority at JJP for all of those two years.

And you're wrong - we did make it a priority to find an answer, but we exhausted all initial leads. We put our pencils down and worked on other things that became more pressing. A fresh set of eyes and some renewed determination helped me find a fix.

I understand your frustration, but as you note there was a workaround: Use 5.05. I wasn't aware that it was no longer available on the JJP site. Had I known that, I would have made sure it got restored.

Promoted items from the Pinside Marketplace
$ 279.95
$ 48.00
Cabinet - Other
ModFather Pinball Mods
$ 279.95
Wanted
Machine - Wanted
Blacklick, OH
$ 279.95
$ 279.95
$ 279.95
$ 279.95
8,200 (Firm)
Machine - For Sale
Victoria, BC
$ 11,500.00
$ 279.95
$ 8,500.00
Pinball Machine
PinballSTAR Amusements
$ 279.95
$ 15.00
Playfield - Protection
Pin Mods
From: $ 5.00
Cabinet - Other
Rock Custom Pinball
$ 279.95
From: $ 20.00
Cabinet - Other
Rock Custom Pinball
$ 279.95
10,500
Machine - For Sale
Gainesville, TX
From: $ 40.00
Cabinet - Other
ModFather Pinball Mods
Hey modders!
Your shop name here

Hey there! Got a moment?

Great to see you're enjoying Pinside! Did you know Pinside is able to run thanks to donations from our visitors? Please donate to Pinside, support the site and get anext to your username to show for it! Donate to Pinside