Optimising dynamic scheduling in smart manufacturing through reinforcement learning PhD

In today's rapidly evolving manufacturing world, efficient coordination of tasks/jobs is vital. Unforeseen dynamic events, such as sudden job insertion, machine breakdowns or operator unavailability, can unexpectedly disrupt production plans, leading to inefficiencies. These occurrences often introduce conflicting objectives, underscoring the need for dynamic multi-objective rescheduling methods that effectively balance time constraints and solution quality. Thus, dynamic scheduling, enabling schedules to be promptly adjusted in response to unexpected events, is crucial for maintaining production efficiency. While prior research has explored aspects of dynamic scheduling, the focus has mainly centred on classical shop scheduling and dynamic scheduling with new order insertions. Traditional methods rely on genetic algorithms and dispatching rules, providing near-optimal results but lacking efficiency and long-term efficacy. Challenges also arise in handling the scheduling of collaborative resources (e.g. machines, operators, etc.), while optimising production time, resource utilisation and meet the desired production goals. Achieving optimal scheduling in such scenarios is challenging due to the interdependencies between different resources, the presence of uncertainties and the need to address various conflicting objectives simultaneously. Furthermore, recent literature in the field of dynamic scheduling has witnessed a shift in emphasis towards understanding the influence of human factors and workforce scheduling on manufacturing processes. Factors like worker fatigue and skill levels can significantly impact task efficiency. However, current approaches often overlook these human-centric aspects, limiting their adaptability in coping with uncertain dynamic disruptions within complex manufacturing environments. 

The primary focus of this project is to address the limitations of existing dynamic scheduling approaches in the context of complex manufacturing systems, characterised by unforeseen disruptions and multi-resource collaboration. The study seeks to address the collaborative scheduling of various resources, while considering human factors like worker fatigue and skill levels to minimise production task delays. To address these challenges, the project will focus on developing an innovative dynamic scheduling approach that integrates multi-agent systems and deep reinforcement learning techniques. Multi-agent systems consist of autonomous units that can make decisions collaboratively, allowing quick responses to disruptions. Deep reinforcement learning leverages advanced algorithms and neural networks to learn optimal decision-making strategies from trial and error. By combining these approaches, the project aims to develop a self-organising system that can dynamically adapt scheduling strategies based on real-time data. This approach will consider collaborative resource utilisation and human factors to minimise makespan, optimise resource utilisation and provide real-time responsiveness to unforeseen disturbances. The research will enable flexible decision-making for tasks such as job sequencing, machine selection and worker assignment. This will lead to improved operational efficiency, work satisfaction and cost in the dynamic manufacturing landscape. 

Cranfield University is wholly postgraduate, and is famous for its applied research in close collaboration with Indsutry. At Cranfield, the candidate will be based within the Manufacturing theme at the Centre for Digital Engineering and Manufacturing (CDEM). The Centre hosts cutting-edge simulation and visualisation facilities. The student will have access to high-end computers and digital technologies in the Centre for ontology-based and knowledge-based systems development, Digital twin development, advanced dynamic modelling and simulations, AI, VR, AR developments. The candidate works on his/her research individually and collaborates with other researchers in the field at the Centre.

The anticipated outcomes of this research hold the potential to enhance dynamic manufacturing scheduling and operational efficiency in complex manufacturing environments. Through the integration of deep reinforcement learning and multi-agent systems, the research anticipates the development of an adaptive self-organising solution capable of promptly responding to unexpected disruptions. This approach will not only optimise scheduling strategies based on real-time data but will also consider collaborative resource utilisation and human factors to mitigate tardiness in production tasks. The implementation of this innovative approach is expected to lead to impactful results: 

• A self-organising multi-agent system will be developed for resource scheduling, enabling manufacturers to swiftly address emergencies and disturbances. This will enhance production responsiveness and adaptability and contribute to reduced downtime, improved resource utilisation and overall optimised production schedules. 
• The development of a deep reinforcement learning model for job sequencing and machine selection will introduce a novel algorithm for tardiness estimation. Incorporating a reward mechanism that combines short-term and long-term returns, the model aims to enhance the convergence and effectiveness of reinforcement learning algorithms. The result will be more robust decision-making processes and efficient scheduling outcomes. 
• Through the development of an attention-based network, a deep reinforcement learning model will be created for worker assignment decisions. This network's innovative approach to effective decision-making is expected to enhance predictive capabilities. It is also expected to enhance the system's responsiveness considering human factors such as fatigue and competencies. 

Through case studies and numerical experiments, the research aims to demonstrate the superiority and effectiveness of the proposed method in addressing complex scheduling challenges in dynamic manufacturing environments. 

This self-funded PhD program offers a range of compelling advantages. It centres on applied research that not only advances your academic journey but also contributes to solving real-world challenges. The program offers diverse training experiences, both internally and externally, enriching your skill set and expanding your knowledge base. Pursuing this PhD at Cranfield University, renowned for its academic excellence, holds the potential to unlock promising career pathways. Moreover, the opportunity to interact with experts from academia and industry not only fosters extensive networking but also offers exposure to cutting-edge insights. This collaborative environment nurtures personal growth and equips you with valuable connections within your field. 

The student will gain from the experience in numerous ways, whether it be transferable skills in the technical area of optimisation and machine learning, or soft skills including presentation skills, project management, and communication skills. There are also numerous employability opportunities that the PhD will offer whether it be in Industry or in Academia.