An unmanned aerial vehicle (UAV)-enabled, big data-driven automatic crop disease diagnosis tool for precision farming
Rice: Malaysians’ staple food
Rice production in Malaysia has reached an approximate value of 2.5 million tonnes in 2021.1 Hence, two mechanisms have to be utilized to meet the demand, i.e. domestic production and importation. Approximately 65-70% of the need is fulfilled from domestic production, where 70% of domestic production are fulfilled by eight main rice fields and the remaining by outside the main fields. Such situation has contributed to Malaysia attaining 72% of self-sufficiency level of rice production in 2019, falling short of 3% than the targeted 75% by Ministry of Agriculture and Food Industries (MAFI).2
Although the numbers seem promising, when leant against growing population (annual growth rate of 2.3%) and presence of invisible consumers3, local rice production is incompetence in meeting rising demands. Adding salt to the wound is yield loss due to pests and diseases, where 15-20% of total rice production areas are estimated to be infected. One of such is Bacterial Leaf Blight (BLB) disease that alone can contribute to approximately RM50 million economic loss. When placed with other challenges facing the agricultural industry such as poverty, old generation of farmers, land scarcity and competition with other commodities, paddy production in Malaysia is further impacted. Worsening situation of Malaysian paddy production can be explained via Organization of Economic Cooperation and Development (OECD)- Food and Agriculture Organization of United Nations (FAO) outlook, which estimated paddy production hovering around 1.6 million tonnes between 2023-2030, and human consumption per capita to reach close to 83 kilograms per capita in 2030.4
Technological boost of Malaysian agriculture
With Malaysian agriculture’s output and quality is being greatly impeded, there are various efforts undertaken to sustain the industry. This includes making paddy production one of the focuses of Budget 2022, via more input and output subsidies, besides encouraging proper usage of Rice Check method. This is done to increase current average paddy yield of 3.5
1 Country Briefs: Malaysia, Food and Agriculture Organization of United Nations: https://www.fao.org/giews/countrybrief/country.jsp?code=MYS
2 Abdul Ghani Wahab 2021, Grain and Feed Annual Report: Malaysia, Foreign Agricultural Services, United States Department of Agriculture:
3 Invisible consumers refer to documented and undocumented migrants and refugees in Malaysia, where their rice consumption patterns aren’t fully comprehended. Source: Omar, S and Tumin, S 2019, The Status of Paddy and Rice Industry in Malaysia, Kazanah Research Institute: https://www.researchgate.net/publication/351223058_The_Status_of_the_Paddy_and_Rice_Industry_in_Malaysia
4 Data extracted for Malaysia from: 2022, OECD-FAO Agricultural Outlook 2021-2030, OECD:
tonnes per hectare to 7 tonnes, in order to move beyond 75% of self-sufficiency level by 2025.5
In spite of various applaudable economic measures, Dr Muhammad Shakirin Mispan believes a need for intervention of IR4.0 technologies to further alleviate the situation. His encounter with a serious Bacterial Leaf Blight (BLB) disease infestation in Sekinchan in 2017, that caused chaos in Integrated Agriculture Development Authority (IADA) Barat Laut Selangor rice granary, and difficulties in assessing infestation level, became a turning point to pioneer a research project on technology driven agriculture. Further discussion with Professor Dr Zulqarnain bin Mohamed of Institute of Biological Science, and Dr Liew Chee Sun from Faculty of Computer Science and Information Technology led to identification of a potential partnership that is working on disease detection using image, and grant application to pursue the project. Finally, the project manifested, in collaboration with Prof Dr. Liangxiu Han of Manchester Metropolitan University, United Kingdom, and funded by British Council through Newton-Ungku Omar Fund.
The team of researchers. From left: Tam Sobeih and Prof Dr. Liangxiu Han from Manchester Metropolitan University, followed by researchers from Universiti Malaya: Dr. Muhamad Shakirin Mispan, Prof. Dr, Zulqarnain Mohamed, and Dr. Liew Chee Sun, during site visit in Sekinchan.
The research focuses on solving our motherland’s evident issue of food safety and security in paddy plantation. The goal is to provide rice producers with a quicker, easier, and more reliable method of diagnosing infected fields by using machine-learning techniques from air than by the traditional on-the-ground human assessment. Therefore, the partnership and
5 2021, Budget 2022: Focus on padi and rice industry to achieve self-sufficiency goals, urges Agri Ministry DG, The Star: https://www.thestar.com.my/news/nation/2021/10/27/budget-2022-focus-on-padi-and-rice-industry-to-achieve-self-sufficiency-goals-urges-agri-ministry-dg
grant play a crucial role in enabling knowledge and technology transfer activities to attain the overarching aim of the research project.
Food security via machine learning technology
As majority of diseases exhibit a range of visual symptoms, current practice of rice disease diagnosis in Malaysia mainly relies on human inspection of symptoms in fields, as portrayed in Image 2 below. Such practice is labour-intensive, costly, and time-consuming, leading to inaccuracy and bias observation outcomes. To add on, it would be deemed an impractical action within a large paddy field, as there are inaccessible areas. Therefore, automatic, accurate, early diagnosis and quantification of diseases in rice production are crucial to allow targeted interventions. By deploying timely and appropriate actions, the research project aims to assist the farmers in effectively and efficiently addressing diseases of paddies.
Example of human inspection of paddy infested with Bacterial Leaf Blight (BLB) disease
Together with other involved researchers, Dr Shakirin believes that with IR4.0 technology, detection and decision-making process can be conducted quickly, accurately, and securely compared to conventional surveillance methods that are affected by human bias. Therefore, the researchers and their collaborative partners seek to uncover potential of using Unmanned Aerial Vehicle (UAV)- also known as drone, as a diagnosis tool for crop diseases in Malaysia's paddy fields. This project is developing a mobile application that can detect Bacterial Leaf Blight (BLB) disease in real-time by synthesizing large-scale images taken from the drone, allowing better field management.
Comparison between infected and non-infected fields
Precision agriculture: A solution for better rice production quantity and quality?
Undeniably, digitally connected agriculture businesses are able to garner a richer picture of data-driven insights, better enabling them to acquire greater control over their operations. In light of technological development towards IR4.0, various image-based methods for automated image acquisition and analysis have been proposed to address the aforementioned issue. One of such is image analysis performed with inexpensive, portable systems such as digital cameras or smartphones. Such methodology is apt for disease identification on a specific plant, or small fields with a few plants. As for a larger-scaled utilization, it is a more practical idea to use remote sensing platforms such as satellites - which have been used for crop monitoring.
In a contemporary context, there is an increasing attention to the potential use of UAVs (drones) in agriculture. Not only it has a relatively low cost for monitoring purposes, but it can also operate at low altitudes and capture images at very high spatial resolutions. These criteria make drones a suitable technological device that can contribute to rapid crop health diagnosis and monitoring. Therefore, various government and non-government actors has been pushing forward with agritech developments in Malaysia, complemented with adoption of big data analytics, Internet of Things and other pillars of IR4.0.
Deployment of drone for disease monitoring
Drone in action
Despite encouraging work, it is important to understand that IR4.0 technological adaptation in the agricultural sector has been relatively slow, partly contributed by lack of Internet
connectivity in rural areas and inability of traditional farmers in adopting technology to collect and analyse data. Specifically, drone utilization in agriculture is still in its infancy, especially in Malaysia. Therefore, the researchers have moved a step forwards in building a comfort within potential users of the technologies. Acceptance studies were conducted at various stages to better understand identified users’ interest in utilization of the technology in managing their fields. Besides seminars and workshops to expose potential of drone technology in agriculture to farmers and students, demonstrations to use the developed technology and software were done. These efforts were undertaken with a hope of increasing the acceptance level of intended users and enabling smooth transition when the technology is publicly available. There are also various collaborative efforts under planning to upskill the users in adopting the technology.
Disease detection via machine learning technology
Bacterial Leaf Blight (BLB) disease is one of the major constraints to rice production, where the disease can reduce up to 70% percent of rice production. The major pathogen causing this disease is bacteria called Xanthomonas oryzae pv. oryzae (Xoo). Symptoms caused by BLB, i.e. a yellowing or scorching pattern, is usually evident at the tip of the leaf (see Image 6 below). Causing death of leaf cell and later reducing leaf area, the plants’ photosynthesis process is disrupted, impeding the quantity and quality of the produced rice grains. Orthodox control measure of BLB disease is via chemical control. Nonetheless, it is impracticable due to absence of specific bactericides that can suppress BLB disease’s infestation.
Paddy field heavily infested with Bacterial Leaf Blight (BLB) disease
Therefore, this research project works on developing software that can be used with smartphones or tablets capable of processing aerial images taken from any commercial drone. Through machine learning methodology, the software compares captured photographs with archival images taken by the drone and provide a direct diagnosis of the BLB disease status in the paddy fields. Adapted from a disease detection mechanism on wheat crops that relies on its users to capture photographs of plants directly, the current version is upgraded to lessen direct human involvement in the process, through detection of diseases from the air. Although development of robust and automatic procedures for image acquisition and analysis with UAVs is a challenging task, it is envisaged that earlier diagnosis and intervention will improve crop survival rates and boost harvests – cutting waste and producing better yield.
Advancement of “big data”
This project fully utilises big data, image processing, machine learning, artificial intelligence, and cloud computing. Software or systems that are automated, accurate, and fast in providing early diagnosis of crop diseases is important to carry out targeted interventions. This collaboration has paved the way for precision agriculture and digital smart farming in Malaysia. The project will develop a novel automated system capable of diagnosing and assessing the severity or level of bacterial leaf blight infestation in paddy fields accurately and rapidly through analysis of the mass number of large-scale images taken from drones to smartphones. This will help in the early prediction and management of disease spread in a more planned manner in paddy cultivation areas in Malaysia.
Drone image analysis
Solutions to farmers’ problems
Evidently, the project aims to appease a certain group or help shed the burden for some individuals. For the use of drones in agriculture, the intended stakeholders or target groups
are the management of rice fields nationwide, particularly those who belong to the government sector. This includes (but is not limited to) the Integrated Agriculture Development Authority (IADA), and the Muda Agriculture Development Authority (MADA). On top of that, this project also involves collaboration with local farmers and agricultural-based non-governmental organizations.
At the moment, the project has successfully developed a prototype of a mobile app using a cloud-based platform that can automatically detect BLB in the field. The app’s accuracy rate is 92% (at the time of writing) but the researchers are working on some adjustments to significantly reduce or remove altogether noise and false positives from the images. Following this, the next step required is to optimize the disease segmentation model and therefore embed it in the mobile app, providing the user with segmentation results even when the Internet is not available. Additionally, the researchers intend to further improve the model for it to work with lower resolution images. This is so that they can map overlay according to the frames extracted from the drone video streaming during the flight, as shown in Image 8. Doing this will inherently speed up the disease monitoring process, as it reduces the dependency on taking images frame by frame. Therefore, faster results and wider coverage can be obtained within a short period of time.
Drone image of fields heavily infected with Bacterial Leaf Blight (BLB) disease
Whether it’s small farmers or large entrepreneurs - rice producers can utilize this technologically-driven solution in the near future so that they can increase the productivity of paddy production while simultaneously reducing financial and labour costs. On top of that, even agencies at the public sector level such as Mardi, Department of Agriculture, Muda Agricultural Development Authority (MADA), and Integrated Agricultural Development Area (IADA) can leverage these technologies to help them provide better services to farmers. Moving beyond merely providing solutions, the government can use the technological findings from this research to formulate more concrete and holistic plans to help farmers and subsequently integrate them into their policies, encouraging the usage of the latest technologies for agricultural development in general and paddy production in particular.
Concurrently, this project will continue to ‘train’ this software to enable it to detect diseases and symptoms of other pests for various crops such as corn and oil palm. The researchers plan to further develop additional features to the software such as diagnosis from moving visual materials such as video and the production of an interface that allows the software to control the drone remotely while analysing images directly. This research will culminate in comparing and finding correlations between various other data sources such as satellite images, smartphones, digital cameras, and other relevant data into one system to address the global challenges of food security.
Regardless of advancements, no research or project comes without its fair share of challenges. One of the most prominent problems or setbacks for this project is that the technology is currently being developed to only identify one particular disease. It is far-fetched to expect a comprehensive project that can cover every agricultural problem or challenge that comes about, as there are infinite issues or possibilities in relation to crop health. Nevertheless, the development of this technology can possibly be expanded to cover other crop diseases as well. With that in mind, it is hoped that drones in agriculture can be further developed to address a multitude of issues within crop plantations, including other notable diseases, crop health in general, as well as crop improvement techniques.
As the project funding is nearing its end, the brains behind the project are seeking more funding to ensure the continuity of the research. Not only that, it is also hoped that they are able to expand their research to cover or address other concerns in the rice agro-ecosystems, on top of their work on the monitoring of BLB. However, the research remains in progress and more data are currently being collected to improve and perfect the technology moving forward.
Agriculture 4.0: the way moving forward
At the moment, the current technology uses RGB images - something that can be obtained from any commercial drone without the need for modification. In order to improve this, the technology can instead use hyperspectral and/or multispectral images. Hyperspectral or multispectral images also can be used to analyse plant physiology-associated data. Traditionally, plant physiological data is usually obtained by using destructive methods. This is because laborious and destructive methods are required to measure chlorophyll content in plants, eventually leading to a high variability outcome. Therefore, the future of the plant physiology study encourages the development and usage of much simpler and instant techniques. In this case, multispectral imaging can provide the much-needed insights on physiological-related inputs including plant photo pigmentation as well as its water content. This can be done through analysing its reflectance at a visible wavelength, which gives physiological data of the plant. Each of the spectral areas delivers and interprets useful data about the plant.
These are just a few of the possible technologies that can be further advanced to develop the Malaysian agricultural scene. While manual crop health checking was previously depended on for precision, it is extremely imperative to adapt to technological advancements and adopt smart solutions to reduce the workload of farmers and not to mention costs that come with traditional labour. Soon, technology can be utilized for precision intervention - where farmers can solve issues in real time when the problems are detected. On top of that, in line with the Internet of Things, a cloud supply chain would be extremely helpful for producers and farmers to share information on the availability of the produce, especially in times of food insecurity due to climate change. Directly engaging producers with consumers through a less complex manner will indirectly reduce the “middle-man” effect where the price will be affordable to the customers while not burdening the cost by the farmers.
Please visit https://agrionefarm.wordpress.com/ for more updates about this project. This project is supported by Newton Fund Institutional Links grant, ID 332438911, under the Newton-Ungku Omar Fund partnership. The grant is funded by the UK Department of Business, Energy and Industrial Strategy (BEIS) and the Malaysian Industry-Government Group for High Technology and delivered by the British Council. For further information, please visit https://www.newton-gcrf.org/newton-fund/
Associate Prof. Dr. Muhamad Shakirin Mispan, Institute of Biological Sciences, Faculty of Science (email@example.com), https://umexpert.um.edu.my/shakirin.html
Co-member: Dr. Liew Chee Sun, Department of Computer System & Technology. Faculty of Computer Science & Information Technology (firstname.lastname@example.org), Professor Dr. Zulqarnain bin Mohamed, Institute of Biological Sciences, Faculty of Science (email@example.com)
Authors: Najla Mohd Bustaman (firstname.lastname@example.org) & Sharumathi M Kavi Rajan (email@example.com)