There isn’t anything to specifically update the favicon of a tab. But you can do the exact same thing websites can update their favicon with; run a content script on the web page that modifies the favicon. Firefox will then show whatever the new icon image is in the tab bar.

I don’t have a better example but the content script of an extension I wrote updates the favicon during media playback in a tab. As an overview it creates a canvas of the original favicon, creates a new link element as the “new favicon”, updates that canvas during playback, and then sets the created link element href attribute to the data uri created from the canvas.

I know you can do this because of the AWS Favicon Update add-on. My advice would be to take a look at how it does it by downloading the XPI from the versions page and unziping its content.

Edit: I did so and its mechanism is fairly basic: it’s a content script that replaces the favicon’s URL in the link tag in the document.