Update failed, Amplipi no longer running

I told the WebUI to go ahead and do the 0.1.9 upgrade; it ended saying it failed and “Failed! Retry” red button showed up. I clicked it, nothing happened, and now it doesn’t seem Amplipi software is running.

I’ve SSH’d in but not sure what to look for/reinstall. Any guidance?

Oh no sorry to hear that! We definitely need a better procedure for handling failures like this. Without a lot of back and forth the simplest fix for this is to re-image your pi: re-imaging your AmpliPi

Before doing that you may want to copy your configuration file off of the pi. It is located in /home/pi/amplipi-dev/house.json.

Let us know if that works. We can also diagnose this by looking at the system logs.

Sorry for the delay, been a bit before I had a chance to get time to work on this. I’ll happily provide logs and see if you can diagnose what went wrong, I can live with my music on headphones for a bit =)

So for starters, house.json was empty; house.json.bak had what I needed.

What log files/etc do you want/need to poke at? I’d also be up for a Zoom session or anything to screenshare and let you poke around.

The update to 0.1.9 changes some things about logging, so I don’t know exactly where this is at. After the 0.1.9 update you should be able to access the system logs at http://amplipi.local:19531/entries. This may or may not be available to you. If it isn’t the same log can be saved over ssh then copied using scp.

Here’s what you need to do over ssh:

journalctl > log.txt

If you are using linux, this can be copied to your system using scp:

scp pi@amplipi.local:log.txt .

Otherwise you can use a program like winscp to copy the file off of the amplipi

You should now have a shiny log file. You may want to send that log over email to contact@micro-nova.com if it makes more sense.

Just to follow up on this now that it was resolved via email.

The problem was updating from 0.1.9 from the LMS Prerelease with LMS Server which had added 4 virtual sources.

Here was the relevant part of the log file:

Dec 16 15:50:07 amplipi systemd[566]: Started Amplipi Home Audio System.
Dec 16 15:50:16 amplipi authbind[9060]: Traceback (most recent call last):
Dec 16 15:50:16 amplipi authbind[9060]:   File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
Dec 16 15:50:16 amplipi authbind[9060]:     "__main__", mod_spec)
Dec 16 15:50:16 amplipi authbind[9060]:   File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
Dec 16 15:50:16 amplipi authbind[9060]:     exec(code, run_globals)
Dec 16 15:50:16 amplipi authbind[9060]:   File "/home/pi/amplipi-dev/venv/lib/python3.7/site-packages/uvicorn/__main__.py", line 4, in <module>
Dec 16 15:50:16 amplipi authbind[9060]:     uvicorn.main()
Dec 16 15:50:16 amplipi authbind[9060]:   File "/home/pi/amplipi-dev/venv/lib/python3.7/site-packages/click/core.py", line 1137, in __call__
Dec 16 15:50:16 amplipi authbind[9060]:     return self.main(*args, **kwargs)
Dec 16 15:50:16 amplipi authbind[9060]:   File "/home/pi/amplipi-dev/venv/lib/python3.7/site-packages/click/core.py", line 1062, in main
Dec 16 15:50:16 amplipi authbind[9060]:     rv = self.invoke(ctx)
Dec 16 15:50:16 amplipi authbind[9060]:   File "/home/pi/amplipi-dev/venv/lib/python3.7/site-packages/click/core.py", line 1404, in invoke
Dec 16 15:50:16 amplipi authbind[9060]:     return ctx.invoke(self.callback, **ctx.params)
Dec 16 15:50:16 amplipi authbind[9060]:   File "/home/pi/amplipi-dev/venv/lib/python3.7/site-packages/click/core.py", line 763, in invoke
Dec 16 15:50:16 amplipi authbind[9060]:     return __callback(*args, **kwargs)
Dec 16 15:50:16 amplipi authbind[9060]:   File "/home/pi/amplipi-dev/venv/lib/python3.7/site-packages/uvicorn/main.py", line 371, in main
Dec 16 15:50:16 amplipi authbind[9060]:     run(app, **kwargs)
Dec 16 15:50:16 amplipi authbind[9060]:   File "/home/pi/amplipi-dev/venv/lib/python3.7/site-packages/uvicorn/main.py", line 393, in run
Dec 16 15:50:16 amplipi authbind[9060]:     server.run()
Dec 16 15:50:16 amplipi authbind[9060]:   File "/home/pi/amplipi-dev/venv/lib/python3.7/site-packages/uvicorn/server.py", line 50, in run
Dec 16 15:50:16 amplipi authbind[9060]:     loop.run_until_complete(self.serve(sockets=sockets))
Dec 16 15:50:16 amplipi authbind[9060]:   File "/usr/lib/python3.7/asyncio/base_events.py", line 584, in run_until_complete
Dec 16 15:50:16 amplipi authbind[9060]:     return future.result()
Dec 16 15:50:16 amplipi authbind[9060]:   File "/home/pi/amplipi-dev/venv/lib/python3.7/site-packages/uvicorn/server.py", line 57, in serve
Dec 16 15:50:16 amplipi authbind[9060]:     config.load()
Dec 16 15:50:16 amplipi authbind[9060]:   File "/home/pi/amplipi-dev/venv/lib/python3.7/site-packages/uvicorn/config.py", line 318, in load
Dec 16 15:50:16 amplipi authbind[9060]:     self.loaded_app = import_from_string(self.app)
Dec 16 15:50:16 amplipi authbind[9060]:   File "/home/pi/amplipi-dev/venv/lib/python3.7/site-packages/uvicorn/importer.py", line 22, in import_from_string
Dec 16 15:50:16 amplipi authbind[9060]:     module = importlib.import_module(module_str)
Dec 16 15:50:16 amplipi authbind[9060]:   File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module
Dec 16 15:50:16 amplipi authbind[9060]:     return _bootstrap._gcd_import(name[level:], package, level)
Dec 16 15:50:16 amplipi authbind[9060]:   File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
Dec 16 15:50:16 amplipi authbind[9060]:   File "<frozen importlib._bootstrap>", line 983, in _find_and_load
Dec 16 15:50:16 amplipi authbind[9060]:   File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
Dec 16 15:50:16 amplipi authbind[9060]:   File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
Dec 16 15:50:16 amplipi authbind[9060]:   File "<frozen importlib._bootstrap_external>", line 728, in exec_module
Dec 16 15:50:16 amplipi authbind[9060]:   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
Dec 16 15:50:16 amplipi authbind[9060]:   File "./amplipi/asgi.py", line 34, in <module>
Dec 16 15:50:16 amplipi authbind[9060]:     application = amplipi.app.create_app(delay_saves=True, mock_ctrl=MOCK_CTRL, mock_streams=MOCK_STREAMS)
Dec 16 15:50:16 amplipi authbind[9060]:   File "./amplipi/app.py", line 768, in create_app
Dec 16 15:50:16 amplipi authbind[9060]:     get_ctrl().reinit(settings, change_notifier=notify_on_change)
Dec 16 15:50:16 amplipi authbind[9060]:   File "./amplipi/app.py", line 128, in get_ctrl
Dec 16 15:50:16 amplipi authbind[9060]:     return Api(models.AppSettings())
Dec 16 15:50:16 amplipi authbind[9060]:   File "./amplipi/ctrl.py", line 144, in __init__
Dec 16 15:50:16 amplipi authbind[9060]:     self.reinit(settings, change_notifier)
Dec 16 15:50:16 amplipi authbind[9060]:   File "./amplipi/ctrl.py", line 260, in reinit
Dec 16 15:50:16 amplipi authbind[9060]:     self.set_source(src.id, update, force_update=True, internal=True)
Dec 16 15:50:16 amplipi authbind[9060]:   File "./amplipi/ctrl.py", line 495, in set_source
Dec 16 15:50:16 amplipi authbind[9060]:     raise exc
Dec 16 15:50:16 amplipi authbind[9060]:   File "./amplipi/ctrl.py", line 485, in set_source
Dec 16 15:50:16 amplipi authbind[9060]:     src_cfg[idx] = self._is_digital(input_)
Dec 16 15:50:16 amplipi authbind[9060]: IndexError: list assignment index out of range

For those of you that try this, the fix is to ssh in after the update fails delete the messed up config::

systemctl --user stop amplipi
cp amplipi-dev/house.json.bak old.house.json
rm amplipi-dev/house.json*
systemctl --user start amplipi

The old config will be saved in old.house.json. If you would like to recover that old config:

  1. scp over old.house.json to your system
  2. remove the sources with ids 4-7 from the configuration
  3. Use the web app to reload the edited configuration