Moisture Sensor Data Processing: From Raw Signal to Accurate Measurements and Classification

 

Soil moisture sensors are essential tools for monitoring agricultural and environmental conditions. However, their accuracy can be compromised by various factors. While the impact of each factor may be small, collectively, they can significantly degrade the reliability of sensor measurements.


Unraveling the Data Processing Pipeline: A Step-by-Step Guide

The figure below outlines the steps involved in converting a raw moisture sensor (soil or soilless) signal into usable percentage measurements. The data processing pipeline includes several key steps: temperature compensation to account for temperature-induced variations, noise reduction to eliminate unwanted signal fluctuations, and baseline correction to establish a reference point for dry conditions.


By applying these techniques, the processed data provides a clear representation of the moisture level, facilitating identification and analysis. While signal classification can be a useful tool, its reliability heavily depends on data quality and site-specific conditions, making it an optional step.

The sensor (APAS T1) measures medium water content by detecting changes in its surface's dielectric constant. The sensor's analog output provides a continuous signal representing the moisture level.



Additionally, the digital temperature output is used to compensate for medium temperature variations. However, the raw analog sensor signal is susceptible to noise and temperature variations, making it difficult to accurately correlate sensor readings with actual moisture levels.

Without proper processing, time-series data can be misleading, as it may be difficult to distinguish true moisture signals from background noise.


Signal Pre-processing

This involves initial steps like analog-to-digital conversion, temperature compensation, and noise reduction. The goal is to clean and prepare the raw sensor data for further analysis.


Signal Processing

This includes steps such as baseline calculation, feature extraction, and normalization. These steps consider user settings, including calibration coefficients, medium type (soil or soilless), and installation type (full burial or partial insertion).

By transforming the raw data into a suitable format, these steps prepare the data for the subsequent machine learning algorithm.


Signal Classification

The processed data is fed into a machine learning model, which classifies the signal into different categories (e.g., rainfall, irrigation, normal cycle).


The Complex Relationship Between Soil Moisture and Temperature: A Novel Approach

One of my most important research findings is the intricate non-linear relationship between soil moisture and temperature. Through extensive data analysis and experimentation, I discovered that the relationship between soil moisture readings and temperature is not a simple linear function, but rather exhibits distinct phases, each with its own linear relationship.

To address this complexity, I developed a novel piecewise linear regression model. This model accurately accounts for the varying temperature sensitivity across different soil moisture ranges, resulting in significantly improved accuracy and reliability.

I hope to share more details on the piecewise model in a future article.

Comments

Post a Comment

Popular posts from this blog

Leveraging Microclimate Data, Proximal Canopy Sensing, and Machine Learning for Precise Soil Water Potential Prediction

Image
  Introduction Accurately predicting soil water potential is crucial for optimizing irrigation practices and minimizing water waste in agriculture. This project delves into the critical relationship between high resolution, in-field microclimate measurements, canopy surface temperature , time of day, and soil water content and potential. Building upon our previous work in biophysical modeling , we employ advanced machine learning techniques to investigate the impact of measurement timing on prediction accuracy. Enhanced Prediction with Machine Learning This project utilizes a range of machine learning algorithms, including: Linear Regression:  Establishes a linear relationship between features and target variables. Decision Tree Regressor:  Makes decisions based on a series of if-else questions. Random Forest Regressor:  An ensemble method combining multiple decision trees for improved accuracy and reduced overfitting. Gradient Boosting Regressor:  Iteratively b...
Read more

The Rise of the "Human Data Farms": A Satirical Look at the Future of AI

Image
  The race for Artificial General Intelligence (AGI) has reached a fever pitch, with tech giants vying for dominance in the AI landscape. As Elon Musk has pointed out ( https://www.theguardian.com/technology/2025/jan/09/elon-musk-data-ai-training-artificial-intelligence ), the increasing scarcity of data for training AI models is a significant challenge. But as data becomes increasingly scarce, a new and perhaps more sinister solution is emerging: the rise of the "Data Farms." Imagine a dystopian future where tech companies, desperate for the raw materials needed to fuel their AI models, begin constructing self-contained communities. These "Human Data Farms" would resemble a hyper-surveilled version of The Truman Show, where residents are paid handsomely to live their lives under constant observation. Every interaction, every conversation, every emotion – all meticulously recorded and fed into the insatiable maw of AI algorithms. This scenario, while seemingly absur...
Read more

For CEA Growers: Tips for Substrate Monitoring and Automation

Image
  Based on my communications and meetings with our customers on solutions for substrate monitoring and decision making in controlled environment agriculture (CEA), I have identified a numbers of important areas to briefly discuss:   1) Sensor quantity optimization Most growers do not need a lot of sensors to monitor their grow rooms. There are good ways of determining the optimum number of sensors required for different applications. In addition to an unnecessarily high cost, a large number of sensors in one facility can create serious design, installation and maintenance headaches for different parties involved. More data does not translate into better insight or decision-making unless we know how to deal with “big data”. Large amount of data that grower does not know what to do with leads to frustration and mistrust in the technology.   More data does not translate into better insight or decisions unless we know how to deal with “big data”. Large amount of data that gro...
Read more