Control and evaluation of room interior lighting using digital camera as the sensor

This paper reports the results of measurements performed in a test room to test how digital camera can be used as luminance meter and thus to investigate the lighting control scheme based on the inputs coming from camera. An indoor lighting control scheme which adapts according to the daylight availability is presented in this work. The camera calibration procedure based on High Dynamic Range Imaging technique is used to obtain the camera response function which allows to relate pixel values obtained from the image and the photopic luminance values. Luminance gradient evaluation for the uniformity analysis is discussed for the test room.


Introduction
Lighting practitioners are still in the research for how to create an architectural space with lighting technology that provides visual comfort and uniformity. For the evaluation and control of designed lighting environment a camera based imaging method is discussed. To help, the literature survey suggests: Simulations (model, computer), Empirical experiences, and a camera-aided imaging method is the most suitable ways. A literature survey and development of camera based system for lighting control is focused here. Camera calibration, camera as a sensor for lighting control are the topics for the discussion. The work explained here is measurement of luminance/illuminance from the image sensor camera and how to integrate it with daylight harvesting scheme for interior lighting. Unlike earlier works, this work involves the use of camera which has many advantages over conventional sensors like photo sensors and occupancy sensors. Day light harvesting is much less successful popular lighting control strategy. Lighting represents 17.5% to 34.2% of all electrical consumption in commercial buildings [2] . So the reduction of this is one of the basic need nowadays. So many researchers have considered this as the main problem and developing different lighting control systems like occupancy based on/off switching, daylight linked controls, scheduled light control. To support such systems the sensing technologies used are PIR sensor , dual technologies and photo sensors .It is proved in the reference papers the digital camera can be used to measure luminance and occupancy and thus it can replace the mentioned sensing technologies [1]. The related works in the same area is quoted and discussed in the literature with the methodology. Paper [1] describes about how luminance measurement can be done with digital image. Detailed study of [1] infers the following data. Through traditional measuring devices, it is possible to measure the photometric information point by point. However, these measurements take a long time; due to the measurement uncertainties in the field errors will be more; and from the obtained data it will be difficult to analyze the lighting distribution and variation. Image captured by the digital camera helps to measure the luminance value within large field of view. It includes that luminance of a scene can be accurately measured using the camera, thus it states that building automation can be developed with camera as sensor in replacement of occupancy and photosensor. This paper presents a method for the usage of digital still cameras as luminance meter. To get the luminance variation in any scene multiple exposure value based photos have to be taken which is the main principle in High Dynamic Range Imaging . From the multiple exposure graphs single HDR image and then camera response function is derived here. This work shows that from the pixel values obtained from HDR Photo is corresponding to the accurate luminance value; the significance of spectra on the accuracy of the results. The tools used is Nikon Coolpix 5400 digital camera with a fisheye lens (Nikon FC-E9) with a tripod for the HDRI luminance measurement and Minolta LS110 for the reference luminance measurements [1]. Camera-aided imaging method for lighting application is suggested by Cai [2] : Conventional light meters are not suitable for measurement of uniformity if the light distribution is not the same. This difficulty can be overcame by the usage of digital camera. Since the eyes can only detect luminance variation, for the evaluation of lighting, luminance gradient can be obtained from the image. The significance of luminance gradient evaluation, the measurement rigs , the data treatment ,practical measurement results were discussed . The acceptable average errors with HDR photographies for different surfaces also mentioned here. For color surfaces the average error can be form 9.3 % to 13.1%. High Dynamic Range photography help to measure and interpret the spatiotemporal distribution of luminance in line with visual perception [2]. Newsham G R and Arsenault C D [3] demonstrated that a single Complementary Metal Oxide Semiconductor (CMOS) camera chip can replace the multiple sensors required by conventional systems for daylight harvesting, shading control and occupancy sensing. They have generated a proof of concept system only, and were demonstrated in a single space. Camera based sensors could effectively reduce the cost of automatic lighting controls [3]. Wuller and Gabele [4] presents a method for the replacement of camera as luminance meter .The measurement is not dependent on exposure settings, instead the camera calibration factor .With the help of calibration procedure the OECF was measured .Using the image from the camera RGB values and OECF were used to calculate the luminance value. The error sources which influence the result of the luminance measurement are also discussed. The following camera types were selected for the accomplishments of the main tests. A DSLR camera the Nikon D2X is chosen [4]. The significant advantages using a digital camera for measurement of luminance [4] is pointed out are: The entire scene luminance can be captured by a digital camera. The field of view of the measuring instrument must be smaller than the source for the measurement of luminance. Small area light sources like individual LEDs are difficult to measure with luminance meter. The field of view of camera pixel is 150 times smaller.
The following section is about, the description given by Hiscocks [5], the selection of a luminance standard for camera calibration. Then the author describe the calibration of the camera, the interpretation of the image data and an example. The following topic is considered as one of the main conversion factor for the illuminance calculation from luminance. Illuminance to Luminance: Under the right circumstances, the luminance L of a surface is related to the illuminance E and reflectance ρ by equation (1). = candela/meter 2 (1) where the quantities are L-Luminance emitted from the surface candela/meter 2 , E-Illuminance of light falling on the surface lux, ρ Reflectance [5]. They used the light emitting diode (LED) array to illustrate the process of using a digital photograph to measure the luminance of the individual LEDs. Using the Tek J-16 luminance meter or Minolta Spotmeter this cannot be done because their sensor field of view is much larger than the light emitting source, and a measurement will underestimate the luminance. The array was photographed with the camera in Manual mode to establish the shutter speed, aperture and ISO, keeping an eye on the camera realtime histogram display to ensure that the image was not overexposed [5]. The theoretical background for the development of the system is discussed in the literature. These papers discuss about the calibration and luminance calculation from image. The gap identified with the proposed scheme is that how the obtained luminance can be used for real time artificial light control.

Methodology & system description
HDRI is used for luminance measurement so as to develop an image that closely resembles to what a human eye can see. For this purpose the radiance information is extracted from the set of multiple exposure low dynamic range photographs. The camera needs to be calibrated in order to get absolute luminance for general lighting applications [6,7].Since luminance value corresponding to the pixel value obtained from the image and scene luminance are not equal it is necessary to find the relation between both the luminance values . For the calibration the relation between luminance obtained from the image and the luminance measured by calibrated luminance meter is studied by applying curve fitting technique with 1 st , 2 nd , 3 rd degree polynomials and error comparison study is done, where it is proved the case of 1 st and 3 rd degree polynomials the error involved are comparable [6,8]. Luminance on the task area is calculated from the HDR image and corresponding dimming is performed based on the available daylight and desired lighting level. The progress work towards the problem of replacing conventional sensors or luminance meter by camera for daylight artificial light integrated scheme is discussed in the report. The main intention is to keep the interior illuminance/ luminance of the system at the desired value.

Camera calibration
Luminance distribution of the total viewing field is required for several luminance measuring tasks. HDRI serve as a luminance mapping tool. To use camera as a sensor for luminance measurement, a camera has to be calibrated by any one of the methods available. Calibrations of raw HDR images can be done in the following ways [2]: 1)Vignetting correction ii) Photometric calibration using a calibration factor (CF).Based on the conversion factor method the relation of actual and scene luminance value is obtained here. To obtain the CF integrating sphere or selection of calibration curve for minimum error using color checker chart method can be taken. Minimum error curve is obtained with the color checker chart here and the steps are mentioned for calibration. These options are based on the ranges required to be measured [6]. LDR images taken under different exposure values must be such that the entire dynamic range of the scene is covered. The luminance from the actual scene is proportional to the pixel values from the HDRI image obtained from the camera. So a luminance meter can be replaced by digital camera [5]. Compared with LDR images HDR image contained pixel information that covers a wider dynamic range. As mentioned in [4], when the brightness of the scene was reduced, it was seen that the calibration curve had the same shape but shifted by a factor. The luminance measured by the luminance meter, of a scene depend on exposure value (EV) which depends on shutter time t and f-number N given by, The pixel value luminance is found by the R, G and B values of each pixel approximated to CIE XYZ color space. [6] The R, G and B values are then converted to luminance value Y by the equation (3) = 0.2126 × + 0.7152 × + 0.0722 × which is termed as the luminance obtained from the HDR image. This software luminance is found for all those pixel coordinates for which the actual values of the region are measured by luminance meter [6,9].
Steps for calibration : 1. Load 11 LDR images with different exposure settings 2. Find Exposure Value for exposer bias at 0 step (reference condition) using equation (2)  Calculate the actual luminance by substituting HDRI luminance in equation of Ynew From the obtained luminance value get the illuminance for the workplane using equation 1. This experiment is carried out for the workplane only with an assumption of the reflectance as a constant. After getting the actual illuminance of the work plane using the following control algorithm given in table 1 PWM signal is generated for maintaining the constant illuminance level through he luminaire. For this work, the set the value of illuminance of the test room is taken as 500 lux. This preset value of illuminance is divided into 5 ranges for the controller design. The set value of illuminance is compared to the illuminance of the test room. As the task illuminance falls in different ranges, different dimming levels are obtained and accordingly the brightness level of the LEDs can be altered automatically. Varying control voltage is generated by varying the duty cycle of PWM signal in Arduino. PWM Generation: The duty cycle of the PWM signal is taken as the average power delivered to the LED. Here, the Ardiuno is being programmed by the control algorithm given in table 1 where Etask is the task illuminance. programming is used to generate PWM pulses. The PWM signals are given to the led driver circuit which in turn controls the switching of the LED light. Depending upon the digital output of the comparator that is given to the I/O pins of ardiuno controller, different duty cycles are generated. This generated PWM signal with varying duty cycle is fed to the LED driver circuit. LED Driving/Switching: The PWM output signal from the Ardiuno is fed to the LED driving circuit. The driver can automatically control the lamp brightness depending upon the change in the input PWM signal. For the proposed system, the LED is controlled using a driver which changes the light output with respect to the change in duty cycle of the PWM pulses. In this work, the generated PWM pulses have a voltage of 3.6V (TTL high). Optocoupler TLP250 is used to transfer electrical signals between the isolated circuitsthe Ardiuno and the LED. The Optocoupler has been powered at +12V. MOSFET IRFZ540N is used for fast switching of LEDs.. The PWM pulses are used to adjust the brightness of the LEDs through the driving circuit mentioned in figure 2.

Experimental setup, results & discussions Camera calibration
The basic setup was put up in the dark room of the lighting lab, EEE department, MIT, Manipal as mentioned in figure 3.   The graphs for HDRI vs actual luminance were plotted as shown in figure 5, from curve fitting tool we get the equations for each selected cases. After the graphs are plotted minimum error curve is found out. A single curve is taken as the reference graph. Then the error is calculated for it. Similarly all the remaining graphs are taken as the reference once and errors are calculated. The minimum error curve is taken as the reference .Sample table for the error calculation with 1700 lux as reference and found the error for with 1400 lux equation and calculated the new luminance value using the multiplication factor obtained and calculation is shown below , similarly 42 such comparisons have been done . Figure 5 shows the image illuminance calculated at the first table of the middle row. Similarly a sample plotted graph is shown below. Sample graph for 500 lux case is shown in figure 6. For example considering 1700 lux equation as the reference equation. We found the error for 1400 lux,1100 lux equation and so on. With this reference equation the factor 2 ^(EVnew-EVold) is multiplied to get a new equation for EVnew. It was found out that 500lux curve with EV 7.64 is the minimum error curve. Thus the equation for 500lux is taken as relation between actual and HDRI luminance. For each case the luminance are compared for a range of 50 to 200 cd/m 2 obtained the new luminance equation and the error is tabulated in table 4. 11 LDR images of the test room were taken to get the HDR image which is used for the illuminance calculation of the desired workplane of the room. The error is derived by taking the difference between measured and estimated luminance value.  Case study 1: The following LDR images ( given in right side of figure 7 )are directly sent to from camera to laptop through wifi sd card on a real time basis which is processed in MATLAB to get the HDR image(left side image of figure 7) generated at 1pm. From the HDR image R,G,B value is generated and thus luminance value is obtained ,which is converted to new luminance value from using the multiplication factor obtained by the calibration procedure. From the new luminance value the illuminance is obtained which is the Etask here This information is given to the control algorithm that generates the PWM signal through Arduino to vary the brightness of the light source as shown in figure 8.  in the working area as measured by the camera acting as luminance measurement equipment. The variation in the PWM signal duty cycle is clearly visible in the CRO display of figure 9 to 12 during different timing..

Fig. 5: Desktop illuminance
Luminance gradient for evaluating lighting: -Measurement of the levels and distribution of the light in the same room with digital camera is shown with following simulations which will help for the further analysis of glare for blind control implementation. Luminance map, luminance gradient magnitude and luminance gradient direction from the calibrated HDR image is developed for the uniformity analysis of the test room which can be further used for lighting hazards. For the Fig 7 .HDR image from LDR images using Luminance HDR luminance map, gradient magnitude and direction obtained is shown in figure 13 to 16. Luminance environments of New Seminar Hall, EEE department, AB1 under stable electric light without considering zones/ 12 April 2016 / 1 pm is shown in figure 13 to 16.

Conclusion
A lighting control system that adjusts the artificial light depending upon the availability of daylight based on the inputs from camera as sensor that provides luminance information of the test room is presented here. How digital camera can be used as luminance meter, luminance gradient evaluation for lighting, desktop illuminance verification for the artificial light control, final implementation of the hardware for the validation is done in the test room. We showed that like camera can be used for daylightartificial light integrated scheme effectively since we can obtain the expected illuminance value from the luminance by calculating the reflectance factor with many trials. Integration of occupancy detection, window blind controller with zone based lighting control are the future work of this paper. Comparison of the results with conventional sensor reading and thus the energy auditing for the propose control scheme are considered as the continuation of this work.