anki.txt

What do you NOT need to do deep learning? (list 3 things)	<a href=https://aiquizzes.com/gcp_images/what_you_do_not_need.png><img src=https://aiquizzes.com/gcp_images/what_you_do_not_need.png></a><br><br><a href='https://aiquizzes.com/questions/1'>view answer on aiquizzes</a>
Are neural networks a recent invention? What year was the mathematical model of an artifical neuron developed?	The model of an <a href="https://en.wikipedia.org/wiki/Artificial_neuron">artificial neuron</a> was developed in 1943 by Warren McCulloch and Walter Pitts. <a href=https://aiquizzes.com/gcp_images/artificial_neuron.png><img src=https://aiquizzes.com/gcp_images/artificial_neuron.png></a><br> In 1969 Marvin Minsky and Seymour Papert published <a href="https://en.wikipedia.org/wiki/Perceptrons_(book)">a very important book</a> where they demonstrated that a simple neural network (inputs connected directly to the output layer) would not be able to learn even something as simple and critical as the XOR function! Later in the book they show that with additional layers (single layer is sufficient) a neural network would be able to overcome these limitations (a message that was missed). And so the <a href="https://en.wikipedia.org/wiki/AI_winter">AI winter</a> began. <a href=https://aiquizzes.com/gcp_images/ai_winter.png><img src=https://aiquizzes.com/gcp_images/ai_winter.png></a><br> The history of neural networks goes way back!<br><br><a href='https://aiquizzes.com/questions/2'>view answer on aiquizzes</a>
What are some of the components of a system that can learn as described in <a href="https://mitpress.mit.edu/books/parallel-distributed-processing-volume-1">Parallel Distributed Processing</a>, a book released by MIT in 1986?	<a href=https://aiquizzes.com/gcp_images/parallel_distributed_processing.png><img src=https://aiquizzes.com/gcp_images/parallel_distributed_processing.png></a><br> <br><br><a href='https://aiquizzes.com/questions/3'>view answer on aiquizzes</a>
What can a sufficiently large network with one hidden layer learn?	It has been <a href="https://en.wikipedia.org/wiki/Universal_approximation_theorem">shown</a> that such a network can learn to approximate any mathematical model to any level of accuracy! Even though this is theoretically possible, in practice we need more layers to achieve good performance. More layers == &quot;deep&quot; and this is how we get deep learning! <a href=https://aiquizzes.com/gcp_images/age_of_deeplearning.png><img src=https://aiquizzes.com/gcp_images/age_of_deeplearning.png></a><br> <br><br><a href='https://aiquizzes.com/questions/4'>view answer on aiquizzes</a>
What is a good way to learn to play baseball? Is it by starting to learn about the physics of a parabola and learning how to stitch a ball, or by watching baseball and tossing the ball back and forth? How does this apply to learning deep learning (or anything else)?	Just as we would not teach how to play baseball to a kid (or anyone for that matter!) by teaching them calculus first, we will not learn deep learning in this fashion either! Instead, in the course, we will learn by playing the whole game - learning how to build end to end models that work! Only gradually, after we understand the big picture, will we start going deeper. <a href=https://aiquizzes.com/gcp_images/play_the_whole_game.png><img src=https://aiquizzes.com/gcp_images/play_the_whole_game.png></a><br> <br><br><a href='https://aiquizzes.com/questions/5'>view answer on aiquizzes</a>
What is machine learning? How does it compare to regular programming?	Machine learning, just like regular programming, is a way of telling computers what to do. But instead of telling the computer exactly how to carry out a specific task (say recognizing cats and dogs in pictures), we write a program where the computer can alter some parameters based on the training examples we show it. In this process, called training, it arrives at an implementation that (hopefully) will perform well on the task we ask it to accomplish. <a href=https://aiquizzes.com/gcp_images/what_is_machine_learning.png><img src=https://aiquizzes.com/gcp_images/what_is_machine_learning.png></a><br> <a href=https://aiquizzes.com/gcp_images/what_is_machine_learning_2.png><img src=https://aiquizzes.com/gcp_images/what_is_machine_learning_2.png></a><br> <a href=https://aiquizzes.com/gcp_images/what_is_machine_learning_3.png><img src=https://aiquizzes.com/gcp_images/what_is_machine_learning_3.png></a><br> <br><br><a href='https://aiquizzes.com/questions/6'>view answer on aiquizzes</a>
ML Jargon - what do we call: <ul> <li>the functional form of a model</li> <li>the weights</li> <li>the results</li> <li>the measure of performance</li> <li>the dependent variable</li> </ul> 	<ul> <li>the functional form of a model -&gt; architecture</li> <li>the weights -&gt; parameters</li> <li>the results -&gt; predictions</li> <li>the measure of performance -&gt; loss</li> <li>the dependent variable -&gt; targets</li> </ul> <a href=https://aiquizzes.com/gcp_images/ml_jargon.png><img src=https://aiquizzes.com/gcp_images/ml_jargon.png></a><br><br><br><a href='https://aiquizzes.com/questions/7'>view answer on aiquizzes</a>
What is a positive feedback loop?	A situation in which the more model is used, the more biased data becomes, making the model even more biased, and so forth. <a href="https://youtu.be/_QUEXsHfsA0?t=3821">&quot;Related issue is that of using data as a proxy for what you are after. In all cases, that data that you have is a proxy for some value that you truly care about and the difference between proxy and actual value often ends up being significant.&quot;</a> <a href=https://aiquizzes.com/gcp_images/positive_feedback_loop.png><img src=https://aiquizzes.com/gcp_images/positive_feedback_loop.png></a><br><br><br><a href='https://aiquizzes.com/questions/8'>view answer on aiquizzes</a>
What is the most important thing to do in the top-down approach to learning?	The most important thing is to be able to run a lot of experiments! A list of useful things to do before the second lecture includes: <ul> <li>make sure you can spin up a GPU server, that you can shut it down when it is finished</li> <li>run the code shown in the lecture</li> <li>use the documentation, use the doc function inside juypter notebook</li> <li>do some searching of the fast.ai docs</li> <li>see if you can grab the fast.ai documentation notebooks and try running them</li> <li>read a chapter of the <a href="https://github.com/fastai/fastbook">fast.ai book</a></li> <li>do the questinnaire at the end of the chapter (not everything has been covered yet, answer only the questions that you can)</li> <li>try to get comfortable with running code</li> </ul> <br><br><a href='https://aiquizzes.com/questions/9'>view answer on aiquizzes</a>
What is the difference between a loss and a metric?	Loss and metric are closely related, but the metric is the thing you care about, the loss is the thing your computer is using as the measure of performance to decide how to update model parameters. <a href=https://aiquizzes.com/gcp_images/metric_vs_loss.png><img src=https://aiquizzes.com/gcp_images/metric_vs_loss.png></a><br><br><br><a href='https://aiquizzes.com/questions/10'>view answer on aiquizzes</a>
Does using a validation set guarantee we will not overfit?	Not at all! Every change in hyperparameters, every run of training on our data where we use our validation set to verify results, makes it more plausible that we will overfit to our validation set! In this case, the overfitting would come from picking an architecture, picking training params, that perform well on our validation set but that do not generalize to other unseen data.<br><br> To be rigorous about this, we should set aside a third bit of data, called the test set. This is a part of our dataset that is neither used during the training nor for calculating our metrics. We do not look at this until the whole project is finished<br><br><a href='https://aiquizzes.com/questions/11'>view answer on aiquizzes</a>
Why transfer learning?	When you use transfer learning you can take advantage of all the prelearned features. Researchers, organizations, invest a lot of time and resources to train large models on large amounts of data. Transfer learning allows us to take advantage of the work that has already been done, modify the architecture slightly, and fine tune the model on our problem. <a href=https://aiquizzes.com/gcp_images/pretraining.png><img src=https://aiquizzes.com/gcp_images/pretraining.png></a><br><br><a href='https://aiquizzes.com/questions/12'>view answer on aiquizzes</a>
What is p-value?	We pick some starting point assumption (say, there is no relationship between two variables). We then gather some data with measurements of independent and dependent variables. And then we ask ourselves, what % of the time will we see a relationship of this magnitude or greater by random chance?<br><br>One way to go about calculating the p-value would be by simulation. Another would be to use a mathematical formula to jump straight to this number. <a href=https://aiquizzes.com/gcp_images/p_value.png><img src=https://aiquizzes.com/gcp_images/p_value.png></a><br> <a href=https://aiquizzes.com/gcp_images/p_value_wikipedia.png><img src=https://aiquizzes.com/gcp_images/p_value_wikipedia.png></a><br><br><br><a href='https://aiquizzes.com/questions/13'>view answer on aiquizzes</a>
What do we need to call on a pathlib.Path to delete the file? <a href=https://aiquizzes.com/gcp_images/path_unlink.png><img src=https://aiquizzes.com/gcp_images/path_unlink.png></a><br> 	<a href=https://aiquizzes.com/gcp_images/path_unlink_answer.png><img src=https://aiquizzes.com/gcp_images/path_unlink_answer.png></a><br><br><br><a href='https://aiquizzes.com/questions/14'>view answer on aiquizzes</a>
When is it easier to clean your data - before or after training? Why?	It is easier to clean your data after initial training, especially using the wonderful tools fastai provides. This way you can use the predictions from your model to help with this otherwise tedious and labor intensive task! <a href=https://aiquizzes.com/gcp_images/image_classifier_cleaner.png><img src=https://aiquizzes.com/gcp_images/image_classifier_cleaner.png></a><br><br><br><a href='https://aiquizzes.com/questions/15'>view answer on aiquizzes</a>
What are the two common problems with regards to data your model can encounter in production?	Out of domain data and domain shift.<br><br><a href='https://aiquizzes.com/questions/16'>view answer on aiquizzes</a>
Can you collect unbiased data?	No! All data is biased. One way of addressing this is writing down a lot of details on the data you are collecting, how it was gathered, in which context it is appropriate to use, how it is maintained, etc. You will not totally eliminate bias, but you will become much more aware of the attributes of your dataset and you increase the chance of not being blindsided by them later.<br><br><a href='https://aiquizzes.com/questions/17'>view answer on aiquizzes</a>
You cannot know the entire behavior of your neural network. A way of addressing this is devising a very careful deployment startegy. What could be the three steps of such a process?	<a href=https://aiquizzes.com/gcp_images/deploy_ml_models.png><img src=https://aiquizzes.com/gcp_images/deploy_ml_models.png></a><br><br><br><a href='https://aiquizzes.com/questions/18'>view answer on aiquizzes</a>
Do you need to use some specific fast.ai functionality to improve your model based on new data? What technique can be leveraged to great success in this context?	You do not need anything special - its basically just a transfer learning problem! Its a fine tuning situation where your pretrained model is yesterday&#39;s model and your fine tuning data is today&#39;s data.<br><br><a href='https://aiquizzes.com/questions/19'>view answer on aiquizzes</a>
What is one of the most challenging things (and why) when deploying ML models?	Feedback loops - because they can take a really minor issue and explode it into a major one. For instance, consider predictive policing. You police the areas the model suggests, you find crime as a natural byproduct of focusing on these areas, you feed that new data into the model and you got yourself a feedback loop! The model is now even more likely to suggest that these areas are to be policed, and if unchecked this model would lead to suggesting the entire police force should focus their efforts on a very small area. To quote <a href="https://rss.onlinelibrary.wiley.com/doi/full/10.1111/j.1740-9713.2016.00960.x">a paper</a> on this subject: &quot;predictive policing is aptly named: it is predicting future policing, not future crime&quot;.<br><br><a href='https://aiquizzes.com/questions/20'>view answer on aiquizzes</a>
When are you at risk of introducing a feedback loop?	When your model, its outputs, are controlling the data that goes into the next training round.<br><br><a href='https://aiquizzes.com/questions/21'>view answer on aiquizzes</a>
In many cases feedback loops are hard to avoid. What is a very good way to alleviate the situation?	Invest in a human in the loop! You still get 90%+ plus economic upside of automating everything, while still providing checks when things are about to go horribly wrong.<br><br><a href='https://aiquizzes.com/questions/22'>view answer on aiquizzes</a>
Why you should start blogging? (list 3 reasons)	<ul> <li>it is a great way of finding jobs</li> <li>it is like a resume, only better</li> <li>!!! helps you learn !!!</li> </ul> <br><br><a href='https://aiquizzes.com/questions/23'>view answer on aiquizzes</a>
When blogging, should you think about the most advanced thing you know and try to write a blog post that would impress Geoff Hinton? Why?	That is generally not a good idea. Most people are not Geoff Hinton. Trying to blog for someone who has more expertise than you is not likely to work well. Another consequence of this approach is that you have a small audience now - there is far more people who are not familiar with deep learning than who are.<br><br>The best advice? Try writing something that you from 6 months ago would find super interesting and very useful! <a href=https://aiquizzes.com/gcp_images/what_to_blog_on.png><img src=https://aiquizzes.com/gcp_images/what_to_blog_on.png></a><br><br><br><a href='https://aiquizzes.com/questions/24'>view answer on aiquizzes</a>
Why should you start each project with creating a baseline?	Because otherwise you will have no indication of how well you are doing! This is also the best way to get familiar with your data. <a href=https://aiquizzes.com/gcp_images/baseline.png><img src=https://aiquizzes.com/gcp_images/baseline.png></a><br><br><br><a href='https://aiquizzes.com/questions/25'>view answer on aiquizzes</a>
How do you reshape tensors in PyTorch?	By using the <code>view</code> method. <a href=https://aiquizzes.com/gcp_images/view.png><img src=https://aiquizzes.com/gcp_images/view.png></a><br><br><br><a href='https://aiquizzes.com/questions/26'>view answer on aiquizzes</a>
What is the difference between a metric and a loss? Which one has a nicely behaved gradient?	The metric is the thing we actually care about, the loss the thing that is similar to what we care about that has a nicely behaved gradient.<br><br><a href='https://aiquizzes.com/questions/27'>view answer on aiquizzes</a>
You have a tensor with <code>requires_grad=True</code>. How do you update its value <strong>without</strong> PyTorch keeping track of the calculation for calculating gradients?	You can achieve this by updating the <code>tensor.data</code> attribute of a tensor!. <a href=https://aiquizzes.com/gcp_images/tensor_data.png><img src=https://aiquizzes.com/gcp_images/tensor_data.png></a><br><br><br><a href='https://aiquizzes.com/questions/28'>view answer on aiquizzes</a>
What is the difference between Gradient Descent and Stochastic Gradient Descent?	Gradient Descent calculates the gradients on the entire dataset at one go. For Stochastic Gradient Descent (SGD) we calculate gradients on batches. Before moving onto the next batch, we modify our model&#39;s parameters based on the gradients. For each iteration through our dataset (which would be called an epoch) Gradient Descent would update the parameters once, SGD would perform as many updates as we have batches.<br><br><a href='https://aiquizzes.com/questions/29'>view answer on aiquizzes</a>
What are the two advantages of presizing, an augmentation method for images?	It performs the computatinally heavy warping of an image on the GPU. Also, through the way it has been implemented, it only performs one interpolation step, leading to the resultant image being of higher quality <a href=https://aiquizzes.com/gcp_images/presizing.png><img src=https://aiquizzes.com/gcp_images/presizing.png></a><br><br><br><a href='https://aiquizzes.com/questions/30'>view answer on aiquizzes</a>
You would like to evaluate how your augmentations work and ensure that everything looks okay. When calling <code>dls.show_batch(...)</code> (where <code>dls</code> is an instance of <code>Dataloaders</code>) what parameter do you need to pass to see a version of the same image augmented in different ways?	<code>unique=True</code> <a href=https://aiquizzes.com/gcp_images/show_batch_unique.png><img src=https://aiquizzes.com/gcp_images/show_batch_unique.png></a><br><br><br><a href='https://aiquizzes.com/questions/31'>view answer on aiquizzes</a>
What do you need to call on an instance of <code>DataBlock</code> in order to obtain a listing of the steps your example undergo as they are loaded and collated into a batch? (this is very useful for troubleshooting).	<code>datablock.summary(...)</code> <a href=https://aiquizzes.com/gcp_images/datablock_summary.png><img src=https://aiquizzes.com/gcp_images/datablock_summary.png></a><br><br><br><a href='https://aiquizzes.com/questions/32'>view answer on aiquizzes</a>
When can feedback loops occur? What are the consequences of feedback loops in the context of recommendation systems?	When your model is controlling the next round of data you train it on. A good example of this are recommendation systems - they predict what content a user will like, but they are also determining what content the user is even exposed to, and helping determine what has a chance of becoming popular. <a href=https://aiquizzes.com/gcp_images/feedback_loops.png><img src=https://aiquizzes.com/gcp_images/feedback_loops.png></a><br><br><br><a href='https://aiquizzes.com/questions/33'>view answer on aiquizzes</a>
AI algorithms (for instance, for determining health benefits) are so resilient to errors they do not need supervision nor a systemic way of identifying and fixing issues. True or false, and why?	The deployment of AI based software requires a good recourse and appeals process. Lack of transparency and oversight on life affecting decisions can bring forth a lot of suffering. A good way of monitoring the performance of an AI based system along with effective processes for making corrections are a must.<br><br><a href='https://aiquizzes.com/questions/34'>view answer on aiquizzes</a>
What are 3 examples of unintended consquences of the tech that you build?	When releasing new tech, the thing to consider is how your tech could be used: <ul> <li>by trolls, harassers</li> <li>by authoritarian governments</li> <li>for propaganda or disinformation</li> </ul> <a href=https://aiquizzes.com/gcp_images/unintended_consequences.png><img src=https://aiquizzes.com/gcp_images/unintended_consequences.png></a><br><br><br><a href='https://aiquizzes.com/questions/35'>view answer on aiquizzes</a>
In the Volkswagen diesel cheating case, who was the first person to end up in prison?	Not a board member, not the CEO, not the managers who coordinated the efforts - it was an engineer. They were following orders from what their boss told them to do, but that is not a good excuse for doing something that is unethical.<br><br><a href='https://aiquizzes.com/questions/36'>view answer on aiquizzes</a>
How does bureaucracy assign responsibility and why it is relevant to AI?	In the words of danah boyd &quot;It&#39;s always been a challenge for a bureaucracy to assign responsibility, bureaucracy is used to avoid responsibility&quot;. Today&#39;s algorithmic systems are often extending bureaucracy.<br><br><a href='https://aiquizzes.com/questions/37'>view answer on aiquizzes</a>
Is ethics culturally dependent?	Yes.<br><br><a href='https://aiquizzes.com/questions/38'>view answer on aiquizzes</a>
What is algorithmic colonialism?	It is a field that studies what are some of the issues when you have technologies built in one particular country and culture being implemented half way across the world in very different cultural contexts often with very little to no input from people living in that culture.<br><br><a href='https://aiquizzes.com/questions/39'>view answer on aiquizzes</a>
Can you absolutely trust the data you are gathering? Why?	No, because data can contain errors, some that can be very subtle. That is why you need to work on maintaining and improving your data. Also, as a person subjected to the workings of the algorithms, very often you will not be able to learn what was the data that the agorithm made its decision on (and thus you will not be able to verify if that data is correct or not)! <a href=https://aiquizzes.com/gcp_images/data_errors.png><img src=https://aiquizzes.com/gcp_images/data_errors.png></a><br> <a href=https://aiquizzes.com/gcp_images/credit_errors.png><img src=https://aiquizzes.com/gcp_images/credit_errors.png></a><br> <a href=https://aiquizzes.com/gcp_images/garbage_in_garbage_out.png><img src=https://aiquizzes.com/gcp_images/garbage_in_garbage_out.png></a><br><br><br><a href='https://aiquizzes.com/questions/40'>view answer on aiquizzes</a>
Why is it important to understand the issues around the use of metrics to track performance?	Because much of AI/ML centers on optimizing a metric. Understanding the issues that can arise when pursuing improvement of a metric is fundamental to rolling out AI/ML systems. The more we know about this area, the higher the chance our system will not act in a completely surprising (and ofen counterproductive way) and if it does, that we will be able to pick up on this behavior and implement countermeasures. <a href=https://aiquizzes.com/gcp_images/problem_with_metrics.png><img src=https://aiquizzes.com/gcp_images/problem_with_metrics.png></a><br><br><br><a href='https://aiquizzes.com/questions/41'>view answer on aiquizzes</a>
What are our online environments designed to be like?	<a href=https://aiquizzes.com/gcp_images/online_environments.png><img src=https://aiquizzes.com/gcp_images/online_environments.png></a><br><br><br><a href='https://aiquizzes.com/questions/42'>view answer on aiquizzes</a>
What are the potential sources of bias in your data / modelling pipeline?	<ul> <li>evaluation bias</li> <li>aggregation bias</li> <li>historical bias</li> <li>representation bias</li> <li>measurement bias</li> </ul> <a href=https://aiquizzes.com/gcp_images/sources_of_bias.png><img src=https://aiquizzes.com/gcp_images/sources_of_bias.png></a><br><br><br><a href='https://aiquizzes.com/questions/43'>view answer on aiquizzes</a>
What is representation bias?	When the performance of the model vary widely across some important feature of the data it will be run on. Often, the source of this bias is not having a diverse enough dataset used for training. <a href=https://aiquizzes.com/gcp_images/representation_bias.png><img src=https://aiquizzes.com/gcp_images/representation_bias.png></a><br><br><br><a href='https://aiquizzes.com/questions/44'>view answer on aiquizzes</a>
Often unethical behavior is driven by management - what can an individual engineer do?	For many of us working in tech, we tend to have a lot of options, often more options than we realize. The are so many companies hiring in tech and this is a leverage that can be used. Employee organizing movements are very promising. Vetting the ethics of a company you are joining is a good practice. And the ultimate weapon in your arsenal - walking away and pursuing other opportunities.<br><br><a href='https://aiquizzes.com/questions/45'>view answer on aiquizzes</a>
What is evaluation bias?	When the common datasets used for evaluating research / tools themselves exhibit representation bias. When the benchmark that is widely used has bias, it is replicated at scale. <a href=https://aiquizzes.com/gcp_images/evaluation_bias.png><img src=https://aiquizzes.com/gcp_images/evaluation_bias.png></a><br><br><br><a href='https://aiquizzes.com/questions/46'>view answer on aiquizzes</a>
What is historical bias?	It is a fundamental, structural issue with the first step of the data generation process and can exist even given perfect sampling and feature selection. Gathering more data is usually not the way to fix it as the problem is with the method that generates the data, that mechanism is biased (for instance, the justice system - if it is biased, anything we trained on its outputs will inherit the bias).<br> <br>One way to mitigate this would be to talk to domain experts and those impacted. <a href=https://aiquizzes.com/gcp_images/historical_bias.png><img src=https://aiquizzes.com/gcp_images/historical_bias.png></a><br><br><br><a href='https://aiquizzes.com/questions/47'>view answer on aiquizzes</a>
What is measurement bias?	It occurs when we measure what we already have measurements for or measure what is easy to measure, and not measure what would need to be measured to paint a complete picture. <a href=https://aiquizzes.com/gcp_images/historical_bias.png><img src=https://aiquizzes.com/gcp_images/historical_bias.png></a><br><br><br><a href='https://aiquizzes.com/questions/48'>view answer on aiquizzes</a>
Does even a bit of diversity help?	Yes. The more diverse a team, the stronger it is, but it makes sense to fight even for the tiniest bit of improvement in the right direction as it can go a long way.<br><br><a href=https://aiquizzes.com/gcp_images/racial_bias.png><img src=https://aiquizzes.com/gcp_images/racial_bias.png></a><br><br><br><a href='https://aiquizzes.com/questions/49'>view answer on aiquizzes</a>
Why does algorithmic bias matter?	<ul> <li>machine learning can amplify bias - it is not only encoding it, but it can also make it worse</li> <li>algorithms are used differently than human decision makers</li> <li>machine learning can create feedback loops</li> <li>technology is power - with great power comes great responsiblity</li> <a href=https://aiquizzes.com/gcp_images/ml_can_amplify_bias.png><img src=https://aiquizzes.com/gcp_images/ml_can_amplify_bias.png></a><br> <a href=https://aiquizzes.com/gcp_images/algorithms_are_used_differently.png><img src=https://aiquizzes.com/gcp_images/algorithms_are_used_differently.png></a><br> <a href=https://aiquizzes.com/gcp_images/why_algorithmic_bias_matters.png><img src=https://aiquizzes.com/gcp_images/why_algorithmic_bias_matters.png></a><br> </ul><br><br><a href='https://aiquizzes.com/questions/50'>view answer on aiquizzes</a>
How do we debias our data or ensure it is bias free?	This is not possible. All data contains bias. The most important thing is to understand how your dataset was created and what its limitations are so that you are not blindsided by that bias, but you are never going to fully remove it. <a href=https://aiquizzes.com/gcp_images/why_algorithmic_bias_matters.png><img src=https://aiquizzes.com/gcp_images/why_algorithmic_bias_matters.png></a><br><br><br><a href='https://aiquizzes.com/questions/51'>view answer on aiquizzes</a>
What is disinformation? (list the three things it involves)	It is not about a single post being true or false - it is an orchestrated campaign of disinformation. It <ul> <li>involves seeds of truth (best propaganda always involves kernels of truth)</li> <li>involves misleading context</li> <li>can involve very sincere people that get swept up in it</li> </ul> <a href=https://aiquizzes.com/gcp_images/disinfo_1.png><img src=https://aiquizzes.com/gcp_images/disinfo_1.png></a><br> Disinformation is an ecosystem - it is not just a single news post or a single story that is misleading or that has false claims in it, it is a bigger ecosystem. <a href=https://aiquizzes.com/gcp_images/disinformation_is_an_ecosystem.png><img src=https://aiquizzes.com/gcp_images/disinformation_is_an_ecosystem.png></a> <a href=https://aiquizzes.com/gcp_images/trumpet_of_amplification.png><img src=https://aiquizzes.com/gcp_images/trumpet_of_amplification.png></a><br><br><br><a href='https://aiquizzes.com/questions/52'>view answer on aiquizzes</a>
What is the new form of censorship?	It is about drowning people out and so it is not necessarily forbidding someone from saying something but totally drowning them out with massive quantity of text, information and comments. AI can really facilitate that and as of now there does not exists a good solution to this. One suggestion by Oren Etzioni is to start using digital signatures as a form of authentication. <a href=https://aiquizzes.com/gcp_images/drowning_people_out_with_comments.png><img src=https://aiquizzes.com/gcp_images/drowning_people_out_with_comments.png></a><br> <a href=https://aiquizzes.com/gcp_images/drowning_people_out_with_AI.png><img src=https://aiquizzes.com/gcp_images/drowning_people_out_with_AI.png></a><br> <a href=https://aiquizzes.com/gcp_images/digital_signatures.png><img src=https://aiquizzes.com/gcp_images/digital_signatures.png></a><br><br><br><a href='https://aiquizzes.com/questions/53'>view answer on aiquizzes</a>
What is Ethical Risk Sweeping?	It is a scheduled risk sweeping activity where a team strives to identify ethical risks, that is choices that may cause significant harm to persons or other entities with a moral status or are likely to spark acute moral controversy for other reasons. <a href=https://aiquizzes.com/gcp_images/ethical_risk_sweeping.png><img src=https://aiquizzes.com/gcp_images/ethical_risk_sweeping.png></a><br> <a href=https://aiquizzes.com/gcp_images/ethical_risk_sweeping_text.png><img src=https://aiquizzes.com/gcp_images/ethical_risk_sweeping_text.png></a><br><br><br><a href='https://aiquizzes.com/questions/54'>view answer on aiquizzes</a>
What is Expanding the Ethical Circle?	It is a practice that encourages the design team (or people working on the machine learning model) to invite stakeholder input and perspectives beyond their own. <a href=https://aiquizzes.com/gcp_images/expanding_the_ethical_circle.png><img src=https://aiquizzes.com/gcp_images/expanding_the_ethical_circle.png></a><br> <a href=https://aiquizzes.com/gcp_images/expanding_the_ethical_circle_text.png><img src=https://aiquizzes.com/gcp_images/expanding_the_ethical_circle_text.png></a><br><br><br><a href='https://aiquizzes.com/questions/55'>view answer on aiquizzes</a>
What is the Think About the Terrible People exercise about?	If the tool you are working on was to be used by people with bad intentions, how could it be leveraged? This activity helps us consider how the tool we are developing could be used by bad actors. <a href=https://aiquizzes.com/gcp_images/think_about_terrible_people.png><img src=https://aiquizzes.com/gcp_images/think_about_terrible_people.png></a><br> <a href=https://aiquizzes.com/gcp_images/think_about_terrible_people_text.png><img src=https://aiquizzes.com/gcp_images/think_about_terrible_people_text.png></a><br><br><br><a href='https://aiquizzes.com/questions/56'>view answer on aiquizzes</a>
What is the objective of the Closing the Loop: Ethical Feedback &amp; Iteration activity?	Its purpose is to develop formal procedures and chains of responsibility for ethical iteration. We need such ongoing practices as ethical design/engineering is never a finished task. <a href=https://aiquizzes.com/gcp_images/closing_the_loop_ethical_feedback.png><img src=https://aiquizzes.com/gcp_images/closing_the_loop_ethical_feedback.png></a><br> <a href=https://aiquizzes.com/gcp_images/closing_the_loop_ethical_feedback_text.png><img src=https://aiquizzes.com/gcp_images/closing_the_loop_ethical_feedback_text.png></a><br><br><br><a href='https://aiquizzes.com/questions/57'>view answer on aiquizzes</a>
Why having a diverse team makes you much more effective?	Because it allows you to capture more unique perspectives and build them into your product / service and thus making it better, more appealing for a broader / target audience. An example mentioned in the lecture is that of Tracy Chou, an early engineer at Quora. Her perspective as a woman led to her identifying and implementing a feature that greatly improved the experience for women using the site. How would have forgoing such a large user segment impacted the growth of Quora? <a href=https://aiquizzes.com/gcp_images/quora_diversity.png><img src=https://aiquizzes.com/gcp_images/quora_diversity.png></a><br> <a href=https://aiquizzes.com/gcp_images/pipeline_vs_diversity.png><img src=https://aiquizzes.com/gcp_images/pipeline_vs_diversity.png></a><br> <a href=https://aiquizzes.com/gcp_images/reasons_women_leave_tech.png><img src=https://aiquizzes.com/gcp_images/reasons_women_leave_tech.png></a><br><br><br><a href='https://aiquizzes.com/questions/58'>view answer on aiquizzes</a>
Why do we need policy (list 4 reasons) AND ethical industry behavior (list 2 reasons)?	Policy is the appropriate tool for addressing: <ul> <li>negative externalities</li> <li>misaligned economic incentives</li> <li>&quot;race to the bottom&quot; situations</li> <li>enforcing accountability</li> </ul> Ethical behavior in industry is necessary as well, since: <ul> <li>law will not always keep up</li> <li>edge cases arise</li> </ul> <a href=https://aiquizzes.com/gcp_images/policy_vs_determining_role_in_genocide.png><img src=https://aiquizzes.com/gcp_images/policy_vs_determining_role_in_genocide.png></a><br> <a href=https://aiquizzes.com/gcp_images/policy_and_ethical_industry_behavior.png><img src=https://aiquizzes.com/gcp_images/policy_and_ethical_industry_behavior.png></a><br><br><br><a href='https://aiquizzes.com/questions/59'>view answer on aiquizzes</a>
What is a <code>Dataset</code> in fastai?	A dataset is something that can be indexed into and that implements the <code><strong>len</strong></code> method (that can have <code>len(...)</code>) called on itself. Once we have a Dataset, we can pass it to a Dataloader. <a href=https://aiquizzes.com/gcp_images/dataset.png><img src=https://aiquizzes.com/gcp_images/dataset.png></a><br><br><br><a href='https://aiquizzes.com/questions/60'>view answer on aiquizzes</a>
What is <code>Datasets</code> in fastai?	A <code>Datasets</code> is an object that groups together a training and a validation dataset. It also accepts a list of transforms. <a href=https://aiquizzes.com/gcp_images/datasets.png><img src=https://aiquizzes.com/gcp_images/datasets.png></a><br><br><br><a href='https://aiquizzes.com/questions/61'>view answer on aiquizzes</a>
In Computer Vision, how are the dimensions of an image given? Does the first dimension correspond to width or height?	In Computer Vision, they do it the other way around to <code>numpy</code> and <code>pytorch</code>. The first dimension is width and the second height (where in numpy we have rows by columns). Hence Full HD is 1920x1080, but the screens are wider than they are tall. <a href=https://aiquizzes.com/gcp_images/cv_image_dim.png><img src=https://aiquizzes.com/gcp_images/cv_image_dim.png></a><br><br><br><a href='https://aiquizzes.com/questions/62'>view answer on aiquizzes</a>
In Deep Learning, what can a matrix lookup be replaced with?	A matrix lookup is an operation in which we retrieve a row or multiple rows from a matrix. Such lookup can be performed by simply indexing into the matrix. All modern Deep Learning frameworks support this operation but that has not always been the case.<br><br>Say we have a number of entities - we can represent each as a one hot encoded vector. When we pass such a representation to our model, it can look up relevant rows of data by performing matrix multiplication with one hot encoded vectors! By representing our entities not with indices, but as one hot encoded vectors, we transform matrix lookup into an operation that can be performed using a mathematical operation!<br><br>Understanding that these two ways of performing lookup are equivalent (with lookup by index being more computationally efficient) is essential - it will form the basis for figuring out what embeddings are and how they work. <a href=https://aiquizzes.com/gcp_images/matrix_lookup.png><img src=https://aiquizzes.com/gcp_images/matrix_lookup.png></a><br><br><br><a href='https://aiquizzes.com/questions/63'>view answer on aiquizzes</a>
Doing a matrix lookup through multiplication with one hot encoded vectors is computationally expensive. What is the name of a computational shortcut?	Embeddings and an Embedding layer! There is nothing special about this - we preserve the ability to backpropagate error to modify embeddings relevant to an example, but the operation is implemented in a more efficent manner than performing matrix multiplication with one hot encoded vectors.<br><br><a href='https://aiquizzes.com/questions/64'>view answer on aiquizzes</a>
How does L2 regularization (weight decay) work?	<p>Regularization reduces the capacity of a model. This makes it less likely that a model will overfit.</p> <p>The capacity of a model is how much space it has to find answers - if it can find any answer anywhere, those answers can include memorizing the dataset. One way to handle this would be to decrease the number of parameters or simplify the architecture in some other way, but generally speaking (especially in the deep learning context) this ends up biasing the model towards very simple shapes.</p><p> There is a better way to do this. Rather than arbitrarily reducing the number of parameters, we instead try to force the parameters to be smaller unless they are really required to be big. </p> <p> We achieve this by adding the sum of all the parameters squared to our loss. </p> <a href=https://aiquizzes.com/gcp_images/l2_reg.png><img src=https://aiquizzes.com/gcp_images/l2_reg.png></a><br><br><br><a href='https://aiquizzes.com/questions/65'>view answer on aiquizzes</a>
You are using the sigmoid trick and the highest value you want your model to be able to predict is 5. What max value do you need to set? (in fastai, the name of the parameter passed to a function that enables this is <code>y_range</code>).	It&#39;s impossible for a sigmoid to get all the way to the top or all the way to the bottom due to its asymptotic nature, no matter how big the x is. If you want to be actually able to predict a value of five then you need to use something higher than 5 as your maximum.<br><br><a href='https://aiquizzes.com/questions/66'>view answer on aiquizzes</a>
You would like to train a Random Forest classifier on data that contains discreet values that have some order (for instance a column size with possible values of small, medium large). What is the name of such a variable? How do you convey this information to <code>pandas</code>?	These are ordinal variables. You can pass this information onto <code>pandas</code> by setting the column to be of type <code>category</code> and passing in the order. <a href=https://aiquizzes.com/gcp_images/categorical_variables.png><img src=https://aiquizzes.com/gcp_images/categorical_variables.png></a><br><br><br><a href='https://aiquizzes.com/questions/67'>view answer on aiquizzes</a>
A key technique to modern machine learning goes by the name of bagging. Who invented it? How does it work?	<p>It was invented in 1994 by Leo Breiman, a retired Berklee professor. The idea is to keep randomly choosing subsets of the training set and training models on them. We store them to use for prediction later. Having accumulated some amount of trained models, we run them on our data and take the mean of their predictions. The process in which we create each subset of a dataset to train on is called bootstrapping (in this sense, bagging is bootstrap aggregation).</p> <p>The reason this works so well, is that despite they models are not trained on the full dataset and are thus not as accurate as they could be, the errors they make are uncorrelated. That means that upon taking the average of the models, the errors cancel out (the average is zero) and we obtain a good result - better than should the prediction come from a single model.</p> <a href=https://aiquizzes.com/gcp_images/bagging.png><img src=https://aiquizzes.com/gcp_images/bagging.png></a><br><br><br><a href='https://aiquizzes.com/questions/68'>view answer on aiquizzes</a>
Does adding more predictors (trees) to Random Forest increase the risk of overfitting?	No! That is the beauty of this algorithm. It is extremely robust to overfitting - training more models on subsets of the training set and taking their mean does not lead to overfitting but generally delivers improved performance (beyond a certain point the gains become very small or cease alltogether). <a href=https://aiquizzes.com/gcp_images/random_forest_overfitting.png><img src=https://aiquizzes.com/gcp_images/random_forest_overfitting.png></a><br><br><br><a href='https://aiquizzes.com/questions/69'>view answer on aiquizzes</a>
What is a partial dependence plot?	It is a plot, constructed from a Random Forest classifier, that describes how a dependent variable varies as we alter one of the independent variables. It aims to hold all other things constant. For a description of how it is constructed please consult the referenced part of the lecture (essentially, we iteratively set the column of interest to all of its unique values, storing results across runs). <a href=https://aiquizzes.com/gcp_images/partial_dependence_plot.png><img src=https://aiquizzes.com/gcp_images/partial_dependence_plot.png></a><br><br><br><a href='https://aiquizzes.com/questions/70'>view answer on aiquizzes</a>
What is the extrapolation problem in the context of Random Forests?	Random Forest is just taking the average of predictions of a bunch of trees and the prediction of the tree is the average of the values in a leaf node (the averages are calculated based on values seen in the train set). Random Forests cannot extrapolate outside of the bounds of the data in the training set. This is a huge problem for things like time series prediction where there is an underlying trend. In a general sense, it is impossible for a Random Forest to extrapolate outside the types of data that it has seen. We need to make sure our validation set does not contain out of domain data. <a href=https://aiquizzes.com/gcp_images/rf_extrapolation.png><img src=https://aiquizzes.com/gcp_images/rf_extrapolation.png></a><br><br><br><a href='https://aiquizzes.com/questions/71'>view answer on aiquizzes</a>
In the context of a Random Forest, how to find out of domain data?	You can create a column <code>is<em>valid</code> which contains 0 for everything in the training set and 1 for everything in the validation set. These are the values we will want our model to predict. The remaining values across the train and validation set become our independent variables.<br><br>We train a Random Forest that strives to predict &quot;is this row from the validation set or the training set?&quot;. If the validation and training sets are from the same distribution, they are not different, than this Random Forest should have zero predictive power. If that is not the case then it means that our validation and training sets are different. To find out the source of that difference we can use feature importance. &lt;a href=<a href="https://aiquizzes.com/gcp">https://aiquizzes.com/gcp</a></em>images/finding<em>OOD</em>data.png&gt;<img src=https://aiquizzes.com/gcp_images/finding_OOD_data.png></a><br><br><br><a href='https://aiquizzes.com/questions/72'>view answer on aiquizzes</a>
What is boosting?	Boosting involves training a small model which underfits your dataset. You calculate predictions using the small model and then you subtract the predictions from the targets (these are the errors that your model is making, they are called <strong>residuals</strong>).<br<br>We then train another simple model, but this time we use the residuals as targets. And we do this over and over again, training a small model, improving the residuals, training another small model, and so on until we reach some stopping criteria (for tree learners, that could be the maximum number of trees). This leaves us with a bunch of models that we do not average, but which we sum. <a href=https://aiquizzes.com/gcp_images/boosting.png><img src=https://aiquizzes.com/gcp_images/boosting.png></a><br><br><br><a href='https://aiquizzes.com/questions/73'>view answer on aiquizzes</a>
Can embeddings learned by neural networks be useful to other types of models (i.e. Random Forests or KNN classifiers)?	Absolutely yes! <a href=https://aiquizzes.com/gcp_images/combining_embeddings_with_other_methods.png><img src=https://aiquizzes.com/gcp_images/combining_embeddings_with_other_methods.png></a><br><br><br><a href='https://aiquizzes.com/questions/74'>view answer on aiquizzes</a>
What is a language model?	A language model is a model where we try to predict the next word of a sentence. A pretrained model can be useful in a similar fashion to how we would use a pretrained Imagenet model - a pretrained model knows a lot about what sentences look like, it might know some things about the world as well. We can leverage all this in the context of our task, irregardless of how much data we have in our training set. <a href=https://aiquizzes.com/gcp_images/language_model.png><img src=https://aiquizzes.com/gcp_images/language_model.png></a><br><br><br><a href='https://aiquizzes.com/questions/75'>view answer on aiquizzes</a>
What are the three approaches to tokenization?	<ul> <li>word-based</li> <li>subword-based</li> <li>character-based</li> </ul> <a href=https://aiquizzes.com/gcp_images/tokenization.png><img src=https://aiquizzes.com/gcp_images/tokenization.png></a><br><br><br><a href='https://aiquizzes.com/questions/76'>view answer on aiquizzes</a>
For NLP, is it better to unfreeze many layers at once or gradually unfreeze layers, one by one?	Gradual unfreezing works better. <a href=https://aiquizzes.com/gcp_images/nlp_gradual_unfreezing.png><img src=https://aiquizzes.com/gcp_images/nlp_gradual_unfreezing.png></a><br><br><br><a href='https://aiquizzes.com/questions/77'>view answer on aiquizzes</a>
What is a stateful RNN?	It is an RNN that maintains state across batches.<br><br><a href='https://aiquizzes.com/questions/78'>view answer on aiquizzes</a>
What is activation regularization (AR)?	It as a type of regularization used for training RNNs. It is very similar to weight decay but rather than adding some multiplier times the sum of squares of the weights to the loss, we add some multiplier times the sum of squares of the activations. In other words, we are not trying to decrease the weights, but decrease total activations. <a href=https://aiquizzes.com/gcp_images/activation_regularization.png><img src=https://aiquizzes.com/gcp_images/activation_regularization.png></a><br><br><br><a href='https://aiquizzes.com/questions/79'>view answer on aiquizzes</a>
What is temporal activation regularization (TAR)?	It is a type of regularization used for training RNNs. It attempts to minimize the difference in activations between adjacent time steps. The idea is that each new word should not change the meaning too dramatically and this tries to model that (that is unless the decrease in loss would make it worth it!) <a href=https://aiquizzes.com/gcp_images/temporal_activation_regularization.png><img src=https://aiquizzes.com/gcp_images/temporal_activation_regularization.png></a><br><br><br><a href='https://aiquizzes.com/questions/80'>view answer on aiquizzes</a>
What is weight tying?	It is a technique where we set the input-to-hidden and hidden-to-output weights to be equal. They are the same object in memory, the same tensor, playing both roles. The hypothesis is that conceptually in a Language Model predicting the next word (converting activations to English words) and converting embeddings to activations are essentially the same operation, the model tasks that are fundamentally similar. It turns out that indeed tying the weights allows a model to train better. <a href=https://aiquizzes.com/gcp_images/weight_tying.png><img src=https://aiquizzes.com/gcp_images/weight_tying.png></a><br><br><br><a href='https://aiquizzes.com/questions/81'>view answer on aiquizzes</a>