Notes on machine learning.https://anil.keshwani.io/https://anil.keshwani.io/blog/information-theory/https://anil.keshwani.io/blog/information-theory/Sketches of some concepts from Information Theory. Readers are referred to Shannon's original 1948 paper A Mathematical Theory of Communication.Mon, 21 Nov 2022 00:00:00 GMTinformation theoryhttps://anil.keshwani.io/blog/the-hierarchical-softmax/https://anil.keshwani.io/blog/the-hierarchical-softmax/The Hierarchical Softmax is useful for efficient classification as it has logarithmic time complexity in the number of output classes, log(N) for N output classes.Thu, 27 Jan 2022 00:00:00 GMTnlpdeep learninghttps://anil.keshwani.io/blog/cpc/https://anil.keshwani.io/blog/cpc/Notes on Representation Learning with Contrastive Predictive Coding (CPC) by Aaron van den Oord, Yazhe Li and Oriol Vinyals.Tue, 23 Nov 2021 00:00:00 GMTself-supervised learningspeechhttps://anil.keshwani.io/blog/self-supervised-visual-representation-learning/https://anil.keshwani.io/blog/self-supervised-visual-representation-learning/This post consolidates several literature summaries from the field of self-supervised visual representation learning.Sat, 20 Nov 2021 00:00:00 GMTself-supervised learningcomputer visionhttps://anil.keshwani.io/blog/four-early-lessons-from-working-on-machine-learning-projects/https://anil.keshwani.io/blog/four-early-lessons-from-working-on-machine-learning-projects/Some high-level reflections from working on a Computer Vision project in PyTorch.Thu, 11 Nov 2021 00:00:00 GMTengineeringhttps://anil.keshwani.io/blog/swin-transformer-pinlab-reading-group/https://anil.keshwani.io/blog/swin-transformer-pinlab-reading-group/I discussed the Swin Transformer: Hierarchical Vision Transformer using Shifted Windows by Ze Liu and colleagues published at ICCV '21 at the PINLab Reading Group on the 3d November 2021.Wed, 03 Nov 2021 00:00:00 GMTreading groupcomputer visionhttps://anil.keshwani.io/blog/lstm-grammar-foreign-language/https://anil.keshwani.io/blog/lstm-grammar-foreign-language/A short explanation of long short-term memory networks (LSTMs), a form of recurrent neural network (RNN) and a breakdown of Vinyals et al.Mon, 06 Sep 2021 00:00:00 GMTnlpdeep learninghttps://anil.keshwani.io/blog/glm-exponential-family/https://anil.keshwani.io/blog/glm-exponential-family/An introduction to the Exponential Family of probability distributions.Mon, 14 Jun 2021 00:00:00 GMTstatisticshttps://anil.keshwani.io/blog/mean-median-mode-representatives/https://anil.keshwani.io/blog/mean-median-mode-representatives/This brief post discusses the underlying basis of the three measures of central tendency that we typically use to represent a distribution, sample or population: the mean, median and mode, prompted by a passing comment made by my Bayesian statistics professor.Wed, 12 May 2021 00:00:00 GMTstatisticshttps://anil.keshwani.io/blog/bayes-conjugate-inference/https://anil.keshwani.io/blog/bayes-conjugate-inference/Bayesian inference for the cases when the data generating process allows for a conjugate setup.Mon, 19 Apr 2021 00:00:00 GMTstatisticsbayesian inferencehttps://anil.keshwani.io/blog/graphs-communities/https://anil.keshwani.io/blog/graphs-communities/We define modularity, Q, as a measure of how well a network is partitioned into communities.Tue, 06 Apr 2021 00:00:00 GMTgraphshttps://anil.keshwani.io/blog/graphs-motifs-and-structural-roles-in-networks/https://anil.keshwani.io/blog/graphs-motifs-and-structural-roles-in-networks/Networks and nodes can be characterised and compared by finding and profiling their network motifs (subgraphs), specifically induced subgraphs and graphlets (connected subgraphs).Thu, 01 Apr 2021 00:00:00 GMTgraphshttps://anil.keshwani.io/blog/jabri-owens-efros-2020-pinlab-reading-group/https://anil.keshwani.io/blog/jabri-owens-efros-2020-pinlab-reading-group/I discussed Jabri, Owens and Efros (2020) Space-Time Correspondence as a Contrastive Random Walk published at NeurIPS at the March 31st session of the PINLab Reading Group.Wed, 31 Mar 2021 00:00:00 GMTreading groupself-supervised learningcomputer visionhttps://anil.keshwani.io/blog/the-probability-distributions/https://anil.keshwani.io/blog/the-probability-distributions/This post introduces a few of the most commonly encountered families of probability distributions: the Poisson, which models rates, and the geometric distribution, which models waiting times for discrete stochastic processes and the exponential, which is a continuous analogue of the geometric relating closely to the Poisson family.Wed, 17 Mar 2021 00:00:00 GMTstatisticshttps://anil.keshwani.io/blog/an-evolutionary-perspective-on-language/https://anil.keshwani.io/blog/an-evolutionary-perspective-on-language/In this post, I look at some aspects of human language that are claimed to be unique to our linguistic communication, as opposed to existing in some analogue or homologue form in the systems that other animals use to communicate.Fri, 24 Apr 2015 00:00:00 GMTbiologylinguisticshttps://anil.keshwani.io/blog/animal-navigation-systems/https://anil.keshwani.io/blog/animal-navigation-systems/Animals display an amazingly reliable, accurate and sometimes mysterious capacity to navigate.Tue, 17 Feb 2015 00:00:00 GMTbiology