Facebook’s graphical representation of algorithmic memory.

Facebook Attempts to Teach Algorithm What to Forget as Means of Conserving Energy

When it comes to a human being’s memory, sometimes we all wish we could forget things. Just as much as we occasionally stumble trying to recall things. It takes a lot of energy and brain cells to remember and forget things. The same is true for algorithms and machine learning programs.

In an article found on venturebeat.com written by Kyle Wiggers, he explains how Facebook has decided that memory is becoming expensive and annoying in other ways. Wiggers writes :

“AI models memorize information without distinction — unlike human memory. Mimicking the ability to forget (or not) at the software level is challenging, but a worthwhile endeavor in machine learning. Intuitively, if a system can remember 5 things, those things should ideally be really important. But state-of-the-art model architectures focus on parts of data selectively, leading them to struggle with large quantities of information like books or videos and incurring high computing costs.”

This can contribute to other problems like catastrophic learning or catastrophic interference, a phenomenon where AI systems fail to recall what they’ve learned from a training dataset. The result is that the systems have to be constantly reminded of the knowledge they’ve gained or risk becoming “stuck” with their most recent “memories.”

Having the ability to go back in time and pull up certain postings or memes that Facebook users had posted even years previously has often produced posting bans for the user. Angry comments are frequently posted on social media about users being busted by algorithms for things they either forgot about or thought were allowed by the Facebook standards at the time.

Facebook’s alternative is Expire-Span, which gradually forgets irrelevant information. Expire_span works by first predicting which information is most important for a task at hand, based on context. It then assigns each piece of information with an expiration date such that when the date passes, the information is deleted from the system.

The Above Shows The Expire-Span Process Of Forgetting

“While this is currently research, we could see the Expire-Span method used in future real-world applications that might benefit from AI that forgets nonessential information,” Facebook wrote in a blog post. “Theoretically, one day, Expire-Span could empower people to more easily retain information they find most important for these types of long-range tasks and memories.”

The detailed story is informative and lengthy.

read more at venturebeat.com