TCD1304 sensor device with reproducible, linear response and 16 bit ADC, interface by SPI.
This repo currently hosts gerbers for our new TCD1304 sensor board which is designed specifically for radiometic linear response and reproducibility. We will be adding design files, gerbers, firmware, and host pc software in Python in the next week or two.
Meawnwhile, the results testing the board are very exciting and we would like to share some of them with you while we are working on updating the repo.
To us this board, you will most likely want the new controller, you can get the gerbers for the controller here. If you want a pre-assembled sensor board or controller, please contact me by direct messaging or email.
As is well known, scientific measurements have to be reproducible; in a practical sense you as well as other scientists should be able to repeat your measurements and obtain similar results. This turns out to be a non trivial challenge for an instrument based on a linear CCD. Researchers who use these instruments will be familiar with effects such as (a) intensity being carryied over from frame to frame or from pixel to pixel, (b) strong peaks being attenuated more than broad peaks, (c) peak height ratios that vary with changes in exposure time or light intensity, and (d) a "mysterious" anomalous response that seems specific to sharp spectral lines. It is easy to understand that these sort of non linear behaviors generate results that are not reproducible.
We maintain that these non linearities originate in electrical design and to some degree how the device is operated, i.e. firmware. And we further propose that the correct solution is to design the hardware to provide the required linearity to produce reproducible outputs.
While we want to "emphasize the positive", we feel some responsibility to comment on one thing. It is certainly true, and easy to see, that the sorts of behaviors described above, some involving attenuation as a function of line width or steepness, some involving charge redistribution, and etc, are not amenable to a simple correction applied one pixel at a time and for some of the behaviors it is not clear that a solution to the inverse problem (i.e., a correction) would exist. It is far easier, and we feel it is more likely to yield reliable, reproducible results, to work with hardware that is designed at the outset to provide the required linearity.
In the following we will show you that the present instrument, based on the TCD1304DG, is in fact pretty linear, and can be used to produce reproducible results. That we are able to achieve linear results, demonstrates that the sensor itself is linear, and that achieving end to end linearity is a matter of good circuit design and firmware.
The present sensor device as noted, is based on the TCD1304DG. The firmware includes a header only library that leverages the FlexPWM of the NXP iMXRT1062 crossover MCU to drive the gates in a reliable controlled way. And the 600MHz ARM7 MCU with 480MHz USB as implemented in the Teensy 4, is able to support frame rates to 100fps for the 3,694 pixel frames produced by the TCD1304DG.
After the introductory material and data, we will explain how all of this works in detail and provide some information to help you get up and running with your own copy of the device.
Since we are going to provide data to compare the present sensor design to commercial instruments lets start with construction of the do it yourself spectrometer that we used for the tests.
The spectrometer assembly is shown in the follow pictures, (a) optical "bench", (b) housing, (c) sensor board, and (d) controller. We have a center wavelength of about 525nm and cover the range from about 300nm to 750nm with a resolution of about 3nm.
The parts list for the above is:
- Grating, 1200 grooves/mm, Thorlabs GT50-12, $250
- 200μm entrance slit, 1 of a set of 6, ebay, ~$40
- Plano Convex lenses (50 to 60mm fl), ebay, ~$20
- SMA905 fitting, Amazon, Digikey, Mouser, Ebay ~$15
- Aluminum plate, Online Metals or Amazon
- Mounts produced with a 3-d printer
- TCD1304 sensor board and controller from this repo, with cables
Detailed discussions on designing a spectrometer are easily found by web search. We will mention a few important highlights.
First, let's choose a center wavelength. For a grating with line density G, the 1st order diffracted wavelength as a function of angle is given by
λ0 G = sin θin + sin θout.
Setting the exit angle to zero (0), our 1200l/mm grating with a center wavelength at 500 nm, gives us an incident angle of about 37 degrees. That happens to be the blaze angle for our grating, So, that works out very well.
The instrument when well aligned should image the slit onto the sensor, when the input is a narrow spectral line. You can use a flashlight as input to align the device. It should look like a well focused rainbow dispersed across the face of the sensor.
The optics have a magnification factor equal to the ratio of the focal lengths of the lenses. Our pixel size is 8μmx200μm, so at 1:1 a 200μm slit makes good use of the pixel height but we give up some resolution. Our resolution limited line width works out to be about 3nm.
The following shows the spectra produced with our spectrometer comapre to that produced by a popular commercial instrument, reportedly an Ocean Optics HR2000 (list price approximately $4,000 to $6,000). The spectrum produced by the commercial instrument can be found here.
Notice that (a) we have slightly better resolution, (b) there are some differences in peak heights and (c) we have a flatter baseline throughout the spectrum. It seems that the Ocean Optics instrument is attenuating strong narrow lines (see for example, the relative height of peak 12 to peaks 6-11, peak 4 to 5, and all of these to peak 3). That behavior in the Ocean Optics instrument might be consistent with a combination of low pass filtering and other effects that we mentioned in the introduction and which will be described later. The poor baseline requires some further investigation, but we see that in a well designed instrument (spectrum on the left) the baseline can be nearly flat. The reader might also notice that we have better resolution and better sensitivity in the blue, perhaps related to the higher density grating and better optical design. Of course another important difference in the two spectrometers is that the cost for the spectrometr on the let can be under $400, i.e., 1/10 of the cost of the commercial instrument.
The equipment list for our linearity study is as follows. Construction of the spectrometer is described here
- Spectrometer built with our new sensor
- Fluorescent lamp to serve as light source
- Neutral density wheel filter for attentuation (individual filters can be used instead)
- 200μm optical fiber
- Miscellaneous mechanicals to hold the lamp, ND filter and fiber in a fixed positions.
Once set up and aligned, the mechanical configuration remains fixed through the duration of the measurements. The ND filter wheel is adjusted and left in a fixed settting throughout a set of exposure setttings.
Lets start with the response of our instrument at three peaks, (a) the smaller broader peak at 487nm, and (b,c) the pair of strong peaks at 542nm and 546nm. In the following note that the y axis is intensity divided by exposure time. For linear response, the intensities divided by exposure time should be nearly constant once the signal is sufficiently above noise. The detector has a noise floor at 0.2mV. We see that in fact the curves are nearly flat apart from the first few points at the shortest exposure times.
Now lets look more closely at how the new sensor device preserves the appearance of spectra and relative peak heights. Here we use less attenuation, the signal is stronger and we can see how the instrument performs near saturation. Some of the peaks are clipped at the longer exposure times. Nonetheless, the result is very reproducible, spectra overlay each other to well within noise, and peak height ratios are very flat except where one of the peaks reaches saturation.
The following are fluorescent lamp spectra collected with another popular commercial CCD spectrometer, the Flame-S. While this is not a model that is currently offered by the manufacturer, it is widely availabe on ebay and it is still cited in in reports involving quantitative results. One recent publication uses this instrument to compare the effectiveness of sun screens.
The manufacturer claims a "corrected linearity" of better than 99.8%. The correction is a simple polynomial in intensity with user specified order and coefficients, i.e., at pixel "p" the "corrected" intensity becomes Ip,c = a0 + a1Ip + a2Ip2 + ...
We might note that the correction requires that the signal at each pixel is independent of other pixels and monotonically increasing with increasing light intensity. Therefore, it remains to be determined whether the correction is effective or valid. For this study, we need to look at the raw output.
In the following it is easy to see that (a) the peak heights are not proportional to exposure time, and (b) relative peak hights vary with exposure time. And, looking at the data closely the response seems non-monotonic.
This repository at present contains the preliminary gerbers, schematic and BOM. We will be adding updated design files, firmware, python code and a detailed explanation of how this works and in particular some insights about the novel issues in achieving linearity for a CCD device used in spectroscopy and scientific imaging.
If you have questions in the meantime, please feel free to contact me.