The primary goals of this experiment are as follows:

• to learn how to evaluate experimental results;
• to learn how to answer the question “How many data points do I need?” by assessing data in real time;
• to assess both statistical and systematic uncertainties in your experiment; and
• to gain experience using Python in the Juypter notebook environment to perform calculations, plot data, and complete least-squares fits.

To remind you, the two models you are testing are as follows.

Assume that a spherical crater is formed by ejecting material; the size of the crater is proportional to the amount of material which was ejected. If the material has a uniform density, then the total mass of the removed material, $M$, is proportional to the volume of the crater, $V$, which is in turn proportional to the crater diameter cubed, $d^3$: $M \propto V \propto d^{3}$. (See Fig. 1.)

Figure 1: Crater geometry

At a minimum, the impactor must provide enough energy to lift the volume of mass completely out of the crater. (See Fig. 2.) If the mass is lifted to a height $h$, the kinetic energy is converted completely to a gain in potential energy of the crater material $U$ as $K = U = M g h$, where $g$ is the acceleration due to gravity.

Figure 2: Lifting the volume of mass out of crater.

Assuming that the crater is spherical, the depth of the crater is proportional to its diameter: $h \propto d$. Using this and the relationship $M \propto d^3$, we have $K = U = Mgh \propto d^4$. Therefore, our first model is that the crater diameter should scale as kinetic energy to the 1/4th power: $d \propto K^{1/4}$.

Assume that a spherical crater is formed by pushing surface material out of the way; the size of the crater is proportional to the amount of material which was pushed away.

Since the material only needs to be pushed out of the way (and not raised up to some height), the energy required is simply proportional to the volume which needs to be moved: $K \propto V \propto d^3$. Therefore, our second model is that the crater diameter should scale as kinetic energy to the 1/3rd power: $d \propto K^{1/3}$.

Using your data from Part 1, your TA will guide you through some exercises to understand how to perform a least-squares fit to compare a full data set to a model. After that, you will use your model to make predictions for larger craters that the class will create (and measure) at the end of the period, and you will try applying your model to real craters found on Earth.

Open up your lab notebook from last week.

• You will continue writing in this document. Do not start a new Google Doc (unless you are part of a new group that did not complete the lab last week)!
• Make a clear mark to indicate where Part 1 notes end and Part 2 notes begin. (You may want to copy the title, date, and student name block and paste it at the start of Part 2.)
• For Part 2 of the experiment, we won't provide prompts… use your judgement to decide what to record in the lab notebook. Remember this is supposed to be a complete picture of what you did… include notes on process, data, analysis, and thoughts/reactions. Don't erase; only add.

Below is the same notebook as last week. We will be moving on to Part 2.

When you make a comparison between a measured number $a \pm \delta a$ and prediction $b$, you can perform a $t^{\prime}$ test to see whether your measured value agrees with that prediction as

where, as discussed in past weeks…

• …$\lvert t' \rvert \leq 1$ is agreement,
• …$1 < \lvert t' \rvert < 3$ is inconclusive, and
• …$\lvert t' \rvert \geq 3$ is disagreement.

Some models, however, have free parameters (also called unknown or undetermined parameters). For example, our two potential cratering models predict that the cratering diameter increases with kinetic energy as either $D = AK^{1/3}$ or $D = BK^{1/4}$, but they don't specify exactly what value the coefficients $A$ or $B$ should have.

How can we make quantitative comparisons in this case?

Fitting data to a model is a way to determine the best values for free parameters. In Part 1, you adjusted the values of $A$ and $B$ in your models until you had what looked to your eyes like the best match to the data. In this part, we will use a more rigorous method for determining the “best” values.

Suppose that you have $N$ data points, where the $i$th data point is $(x_i, y_i \pm \delta y_i)$. You want to see if your data are well-described by a particular function, $f(x)$. Let's suppose, just as an example, that you wanted to test a linear function $f(x) = mx$, where $m$ is the slope of the line. This slope is an example of a free parameter in the model, and we want the fit to tell us what the “best” value is for $m$.

First, let's define a function called the residual. The residual of the $i$th point is $\chi_i$ and is defined as

This form should immediately look familiar: it is the $t^{\prime}$ comparison between the measured value $y_i \pm \delta y_i$ and the value predicted by the function at the same point, $f(x_i)$.

A “good” fit will be one where the value of the free parameters in the fit function make this residual small (meaning that the |$t^{\prime}$| at this point is small, and therefore in agreement). But perhaps values which are good for this point aren't quite as good for the next point… or the point after that. So, rather than focus on a single residual, we want to instead pick values for the free parameters which minimize the sum of all the squares of the residuals,

(We look at the squares of the residuals because we want each contribution to the sum to be positive. If a data point is a little bit higher than what is predicted by the fit function, that should count the same as if the data point is a little low. Squaring accomplishes this.)

It is possible to minimize a function by hand by using calculus, but for this class, we will instead rely on a computer algorithm to find the value for each fit parameter in a model which minimizes the value of $\chi^2$. Such a fit is called a least-squares fit because it finds the fit values that give the least value of the sum of the squares of the residuals.

In such a fit, you supply the data points and the model function, and the fit returns the best values for each fit parameter and the minimum $\chi^2$ value that results when you use those values.

You can now apply the same formalism from above to your question: what are the best values for $A$ and $B$ in the 1/3- and 1/4-power law models?

To do this, return to the Google Colab notebook above, and start working on Part 2. There you will need to enter your data – energy and corresponding average crater diameter with uncertainties – and the code will guide you through the least-squares fits.

Work through the notebook slowly! Try to understand what is happening at each step, and talk to your TA if you don't know what you are looking at.

When the fits are done, you will have values for the best fit parameters and plots showing what the fits look like. The plots should look similar to what you found in Part 1 by-eye, but now we have a more quantitative justification for what is the “best” fit.

In addition to being the thing which is minimized, we can use the final $\chi^2$ value to determine whether our model overall is in agreement with the data or not. (It is still possible for the “best” fit to be a “bad” fit, for example.) For this reason, $\chi^2$ is sometimes referred to as the “goodness of fit” parameter“.

First, note that $\chi^2$ can grow arbitrarily large; if we increase the number of data points used in the fit, we increase the value of $\chi^2$. Therefore, it will help us to look not at $\chi^2$ itself, but at quantity called the reduced chi-squared, $\chi^2_{red}$. If we have $N$ data points and $k$ free parameters in the fit, then the number of degrees of freedom is $\nu = N - k$ and the reduced chi-square is

The reduced chi-square is sort of like the average chi-square per data point, or equivalently the average residual.

Conceptually, what does the reduced chi-square represent and how can we use this value to determine if our model is in agreement with the data or not? Suppose you have one point that is very close to the fit line so that its distance away is less than the size of its uncertainty; for such a point, $\chi_i = (f(x_i)-y_i)/\delta y_i < 1$. Now suppose another point is far away from the line, so that its distance away is greater than its uncertainty; therefore, $\chi_i = (f(x_i)-y_i)/\delta y_i > 1$. If we have a “good” fit, then we'd expect to have some close points ($\chi_i <1$), some far points ($\chi_i >1$), and some medium points ($\chi_i =1$), so we would expect our average residual (i.e. our reduced chi-square) to be about 1.

Let's look at a few scenarios:

• $\chi^2_{red} \approx 1$: the scatter of the data around the fit line is about what you would expect based on the size of the uncertainties. The data and the fit agree.
• $\chi^2_{red} \gg 1$: the scatter of the data around the fit line is greater than you'd expect based on the size of the uncertainties. Either the model does not agree with the data, or the uncertainties on the data points are too small (possibly because there are unaccounted for uncertainties or because of a systematic bias).
• $\chi^2_{red} \ll 1$; the scatter of the data around the fit line is smaller than you'd expect based on the size of the uncertainties. The uncertainties are likely over-estimated or more data is needed to test the model.

Unlike the $t^{\prime}$ test, these are not hard rules about agreement or disagreement. But it can be helpful as part of the discussion about the quality of your fits.

Why do we divide by the number of degrees of freedom, $\nu = N - k$, instead of just the number of points, $N$?

Each time we add another free parameter to the model, we “constrain” the model more. Think, for example, about what happens when you have two data points and you try to fit them to a line $f(x) = mx + b$. We have two data points ($N = 2$) and two fit parameters ($k = 2$), so we have zero degrees of freedom ($\nu = N - k = 0$). The line will go exactly through both points and the chi-squared value will be zero, $\chi^2 = 0$. We have effectively “used up” two data points worth of information to do the fit, so we have no “freedom” left to let the fit wiggle around the data points.

Now consider doing the same fit with three or more data points. The line is now no longer guaranteed to go through each point exactly, and so $\chi^2$ value will no longer be zero.

By dividing $\chi^2$ by the number of degrees of freedom instead of just by $N$, we better account for the information lost (used to constrain the model).

The function you have arrived at is an example of a scaling law.

You may already be familiar with at least one form of the use of scaling laws. Aeronautical engineers construct small scale models of aircraft and test their designs in wind tunnels. If the model is aerodynamically stable, then the scaling nature of the physics involved says that the full size airplane will perform similarly.

In our case, as long as the underlying assumptions of the model remain valid, there is no reason that the functional relationship you have determined should not be valid for craters of all sizes (for impactors of similar density into material of similar density and granularity). Craters on the moon for example should likely follow the same scaling law which applies to your sand craters.

Towards the end of the period, your TA will lead the class into the hall and drop a larger stainless steel ball (from a significantly larger height). You will use your model to predict crater diameters for energies that are orders of magnitude larger than what you studied in Part 1.

• The mass of the ball you will drop is approximately 66.8g.
• The distance from the second floor landing is about 4.68m to the sand surface, and from the third floor landing it is about 8.78m to the sand surface.
• Predict the size of crater produced by such an impactor using your model.
• Provide your TA with your crater diameter estimates and uncertainties.

After the experiment, the TA will call the group together for a discussion of the class findings.

• How did your predictions (from the model) match the measured values?
• Was there much scatter in the predicted values from your classmates or were all groups roughly in agreement?

Below is a Google Maps image of a portion of the Nevada Test Site where over 1000 nuclear weapons tests were conducted. You can see numerous craters formed from both above ground and below ground detonations of nuclear weapons which occurred in the 1950s. On the left side of the image is an impressive crater known as the Sedan Crater which was produced as part of Operation Plowshare to test the feasibility of using nuclear weapons for civilian construction purposes. The crater was produced by the detonation of a $10^{4}$ kiloton ($4.40 \times 10^{12}\text{ J}$) thermonuclear explosion.

Figure 4: The Nevada Test Site with the Sedan Crater marked on the far left. (Image via Google Maps)

Even though the crater was produced by an explosion at the surface near the surface instead of an impactor from space, we want to see if your model holds for this crater.

• Rearrange your scaling law equation so that it becomes kinetic energy as a function of diameter.
• The Sedan Crater has a diameter of 390 m. Using your rearranged scaling law, what would you predict for the energy yield of the nuclear weapon that produced it? Is the value of the same order of magnitude as the known value ($4.4 \times 10^{12}$ J)?
• Using propagation of uncertainties, determine an uncertainty on this energy.
• You may find that your model predicts a value which is the right order of magnitude, but which does not actually agree within uncertainties. Can you think of some reasons why the model may not be correct? (As a hint, think about the dominant process in your model… areas of these craters have been found to be compacted and some of the sand was heated and turned into glass. If some of the energy of the blast went into these processes, what does that mean for your prediction? Will it be too high or too low?)

Another famous crater is Chicxulub, the crater formed by the asteroid that struck the earth around 65 million years ago causing the mass extinction of the dinosaurs (and many other species). Unlike your experiments where the impactor was dropped directly down onto the surface, it is estimated that the asteroid hit the earth at an angle of between 45$^{\circ}$ and 60$^{\circ}$ from horizontal (source: Wikipedia).

Figure 5: Location of the Chicxulub Crater on the Yucatan Peninsula in Mexico. It should be noted that the crater can't be seen on an ordinary map; it took a satellite radar scan to detect the 3-5 m tall edge.

• The Chicxulub Crater has a diameter of about 100-150 km. Using your rearranged scaling law, what would you predict for the energy of the asteroid that produced it?
• Expert estimates – using scaling laws like yours as well as other evidence – suggest the kinetic energy of the asteroid at the time of impact was between $3 \times 10^{23}$ and $6 \times 10^{25}$ J (source: Wikipedia). Why might your prediction not agree with the expert's estimate? Is your prediction too high or too low, and is that consistent with your reasons for what may be causing disagreement?

On December 24, 2021 a meteorite struck Mars near where the NASA InSight Lander is operating. Devices on the craft detected seismic waves that scientists at first thought were due to an earthquake (marsquake?), but satellite imagery from NASA's Mars Reconnaissance Orbiter later showed a new impact crater on the surface consist with the location of the source of the waves.

There are no questions to answer, but you might enjoy this summary of the recent study (published October 22, 2022): https://www.jpl.nasa.gov/news/nasas-insight-lander-detects-stunning-meteoroid-impact-on-mars.