Get Started
image

image Weekly Tech News – Week #9

image Reinforcement Learning in Nuclear Fusion

There exits a research apparatus called tokamak in a nuclear fusion reactor which produces plasma inside it. The goal is to efficiently control the generation of this plasma in order to harvest the nuclear energy. DeepMind handles this task by means of training a reinforcement learning policy in order to steer the very coils inside tokamak responsible for generating the plasma. They have the corresponding a paper published in nature. More details to be found on their official blog.

image Google's Carbon Footprint Owing to Machine Learning

Google gave out its 4M model to reduce carbon emission due to machine learning. They introduced their 4M principle which stands for Model, Machine, Mechanisation and Map Optimization, into which they categorize that various actions that can be taken in order to reduce less energy thereby reduce carbon. It hints several actions such as moving data center with access of green energy, better cooling methods, more efficient machines, and much more. here to learn more about it.

image Systems Learn Like Humans & Animals

Yann LeCun of Meta AI gives the architecture on building next generation autonomous intelligence. The centerpiece of the architecture is the predictive world model. A critical challenge with constructing it is how to enable it to represent multiple plausible predictions. Additionally, how can it learn abstract representations of the world so that important details are preserved, irrelevant details are ignored, and predictions can be performed in the space of abstract representations. LeCun’s vision requires much deeper exploration than is possible in this brief blog post, however it gives a nice summary of what next is coming.

image Vision Models on Uncurated Images

A new paper, by MetaAI shows that vision models are more fair and robust when trained on uncurated images without supervision. This goes against the common believe that images must be preprocessed before training otherwise model outcomes will be highly biased. They collected a humongous amount of images on the internet without any filters or preprocessing and trained their models on them. To their surprise, they notice considerably better performance in terms of bias and robustness compared to the models trained on curated datasets such as image-net.

image Object Sketching Model

A system that comes out of this recent paper, namely, CLIP, uses CLIP (Contrastive-Language-Image-Pretraining) together with differentiable sketches for drawing, and creates sketches of pictures with various levels of abstractions. Hence, not needing any sketch datasets. The approach further supports of use of various brush styles to give sketches more artistic touch.

image Emoji Search Using Transformers

One of the lead applied researchers of OpenAI shared in her recent tweet, her online app for searching emojis. The code is available online which essentially gives a call to an openAI api and using their trained word embeddings to perform emoji search. It's a cool idea on kick-starting your NLP journey.

image NLP with Transformers (Book)

A recent book on learning how to use transformers in NLP is out on amazon, now. Book provides a great start on theoretical foundations as well as code snippets directly from Hugging Face. Both authors are working as learning engineer at Hugging Face.

image Anomalib

Check out the cool Anomalib, a deep learning library that aims to collect state-of-the-art anomaly detection algorithms for benchmarking on both public and private datasets.

image Several New T5 Checkpoints

@GoogleAI open-sourced all of their new efficient T5 checkpoints. All those 200+ T5 checkpoints are exported to Hugging Face here. They shared news in their following tweet here.

image Textless NLP

Textlesslib is a library from Meta AI, aimed to facilitate research in Textless NLP. Textless NLP is an active area of research that aims to extend NLP techniques (and tools!) to work directly on spoken language. The goal of the library is to speed up the research cycle and lower the learning curve for those who want to start.