(no entries)
The goal of this 3-month Forschungspraxis or 6-month thesis is to develop a symbolic task planner based on pretrained multi-modality LLMs and integrate it into existing task and motion planning (TAMP) frameworks.
TAMP has proven effective in addressing a wide range of long-horizon tasks within a single framework[1]. However, the integrated symbolic planner typically depends on manually crafted domain descriptions to operate correctly, which constrains the framework's adaptability. Conversely, LLM excels at reasoning with "common sense" and is resilient to flawed prompt inputs[2]. How could it be used to address the problems in TAMP?
Your tasks:
- Understand the principle of classic TAMP (eTAMP[1])
- Explore the roles of pretrained LLMs in eTAMP: planner or planner assistant?
- Evaluate the proposed methods in benchmark tasks and compare them with baselines
- Document everything
Requirements:
- Experiences with physical simulators (e.g., PyBullet)
- Knowledge of robot motion planning
- Knowledge of PDDL language
- Experience with Git
Contact:
Dr. Tianyu Ren (tianyu.ren@tum.de)
Georg-Brauchle-Ring 60-62, 80992 München
Refenrences:
[1] Ren T, Chalvatzaki G, Peters J. Extended tree search for robot task and motion planning. arXiv preprint, 2021.
[2] Liang J, Huang W, Xia F, et al. Code as policies: Language model programs for embodied control ICRA. IEEE, 2023
To apply:
Send your personal info as attachment to the contact person with the following naming
FirstnameLastname_ Forschungspraxis/Thesis_Staringdate.pdf
e.g., TianyuRen_Thesis_26062024.pdf
Robot Control
The actuator presented in [1,2], based on the Bi-Stiffness Actuation (BSA) concept, consists of five modules, i.e., motor, spring, brake, clutch, and link. With these modules, the actuator can be configured in various operating modes. It also consists of various sensors which enable multiple sensor feedback. The low-level controller currently implemented on this actuator is a cascaded current/velocity/position controller where each loop is PI-based. This cascaded controller is implemented on only the motor module, leaving other modules without low-level controllers.
The objective of this 6-month master’s thesis is to develop a novel Modular Control Framework for Modular Actuator with Multiple Sensor Feedback. By exploiting the modular structure characteristic of the BSA, the effects of each element such as link decoupling, the performance of the elastic element, stiffness brake, and motor control can be tested independently. Moreover, other factors such as friction, hysteresis, effects of coupling and mechanical play between parts, and the effect of switching which creates instantaneous change in velocity can also be analyzed and their effect reduced through control at low-level. To this end, the objective is to consider each module separately and develop a control system that enhances its performance. The end goal is to integrate the controllers into a generalized framework for the combined system.
Requirements for this position
- Dynamic systems modeling techniques
- Background in hardware development and system integration
- Good programming skills with Matlab, Python, and C++
- Experience with system modeling techniques
- Background in robotics with basic understanding of manipulator kinematics and dynamics.
- Working skills in the Ubuntu operating system
- Knowledge of parameter estimation algorithms and sensor fusion frameworks such as Kalman filter is a plus
- Experience in design of experiments is a plus
- E. P. Fortunić, M. C. Yildirim, D. Ossadnik, A. Swikir, S. Abdolshah and S. Haddadin, “Optimally Controlling the Timing of Energy Transfer in Elastic Joints: Experimental Validation of the Bi-Stiffness Actuation Concept,” in IEEE Robotics and Automation Letters, vol. 8, no. 12, pp. 8106-8113, Dec. 2023, doi: 10.1109/LRA.2023.3325782.
- D. Ossadnik et al., “BSA - Bi-Stiffness Actuation for optimally exploiting intrinsic compliance and inertial coupling effects in elastic joint robots,” 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Kyoto, Japan, 2022, pp. 3536-3543, doi: 10.1109/IROS47612.2022.9981928.
- K. Samuel, R. Oboe and S. Oh, “A Reduced-Order Multisensor-Based Force Observer,” in IEEE Transactions on Industrial Electronics, vol. 69, no. 5, pp. 4946-4956, May 2022, doi: 10.1109/TIE.2021.3086719.
To apply, send your CV, short motivation, and transcripts to Samuel Kangwagye (s.kangwagye(at)tum.de).
The actuator presented in [1,2], based on the Bi-Stiffness Actuation (BSA) concept, consists of five modules, i.e., motor, spring, brake, clutch, and link. With these modules, the actuator can be configured in various operating modes. It also consists of various sensors which enable multiple sensor feedback. The low-level controller currently implemented on this actuator is a cascaded current/velocity/position controller where each loop is PI-based. This cascaded controller is implemented on only the motor module, leaving other modules without low-level controllers.
The objective of this 6-month master’s thesis is to develop a novel optimized framework for integration of direct (sensor) and indirect measurements within the modular actuator to improve measurement accuracy while accounting for sensor redundancy. The cases include:
- Parameters that can be estimated indirectly – develop the estimation algorithm(s)
- Parameters that can be measured with sensors – develop the sensor fusion algorithm(s)
- Parameters that can be estimated by combining direct and indirect methods – develop the integration algorithm(s)
Data integration/fusion frameworks such as the Kalman filter, etc., will be utilized. See example in [3].
Requirements for this position
- Dynamic systems modeling techniques
- Background in hardware development and system integration
- Good programming skills with Matlab, Python, and C++
- Experience with system modeling techniques
- Background in robotics with basic understanding of manipulator kinematics and dynamics.
- Working skills in the Ubuntu operating system
- Knowledge of parameter estimation algorithms and sensor fusion frameworks such as Kalman filter is a plus
- Experience in design of experiments is a plus
- E. P. Fortunić, M. C. Yildirim, D. Ossadnik, A. Swikir, S. Abdolshah and S. Haddadin, “Optimally Controlling the Timing of Energy Transfer in Elastic Joints: Experimental Validation of the Bi-Stiffness Actuation Concept,” in IEEE Robotics and Automation Letters, vol. 8, no. 12, pp. 8106-8113, Dec. 2023, doi: 10.1109/LRA.2023.3325782.
- D. Ossadnik et al., “BSA - Bi-Stiffness Actuation for optimally exploiting intrinsic compliance and inertial coupling effects in elastic joint robots,” 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Kyoto, Japan, 2022, pp. 3536-3543, doi: 10.1109/IROS47612.2022.9981928.
- K. Samuel, R. Oboe and S. Oh, “A Reduced-Order Multisensor-Based Force Observer,” in IEEE Transactions on Industrial Electronics, vol. 69, no. 5, pp. 4946-4956, May 2022, doi: 10.1109/TIE.2021.3086719.
To apply, send your CV, short motivation, and transcripts to Samuel Kangwagye (s.kangwagye(at)tum.de).
Brain Computer Interfaces
The potential for invasive Brain Computer Interface (iBCI) systems to dramatically improve the livelihoods of those who have lost motor functionality has increased in recent years. This improvement has emerged due in part to advances in the field of deep learning, but the performance of these models varies significantly depending on the quality of the data collection and the design of the deep learning model architectures. This project focuses on designing and refining deep learning architectures tailored for decoding spiking neural data, with an emphasis on handling sparse and binary input data, continuous output kinematic features, and maintaining a small footprint to facilitate real-time inference. Research with this project will provide experience in pushing the boundaries of what's possible with iBCI systems and will involve working in a collaborate environment for one of the most difficult decoding tasks in the bio-engineering field. Researchers will have access to state-of-the-art facilities and computational resources and will benefit from mentorship opportunities and professional development.
Tasks may include a few of the below (to be discussed depending on your interest and background)
- Further develop and optimize an existing decoding pipeline in python
- Develop state-of-the-art deep learning models based on convolutional and/or recurrent neural networks
- Investigate intermediate latent spaces that may exist between the neural data and the associated kinematic features
- Create a reporting mechanism to make assessment of different architectures efficient, which will enhance our ability to rapidly iterate and improve decoding models
Prerequisites
- Proven experience with building deep learning architectures in Tensorflow
- Proficiency in python
Helpful but not required
- Experience with real-time inference with trained models
- Theoretical understanding of manifolds and dimensionality reduction methods
Position
Current project is targeted for a research internship or semester thesis. It’s not a paid position, nor a master thesis ad.
Related Literature
- Decoding arm kinematics: Paper 1 (https://journals.sagepub.com/doi/full/10.1177/0278364914561535) Paper 2 (https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(12)61816-9/)
- Backend Communication: (https://www.biorxiv.org/content/10.1101/2023.08.08.552473v1.full.pdf)
For more information, please contact:
Dr. Alexander Craik (alex.craik(at)tum.de)
Ioannis Xygonakis (ioannis.xygonakis(at)tum.de)
Robotic hand modeling
Type: Forschungspraxis
Diverse robotic hands have been designed and developed to achieve the human hand's performance in daily tasks. The tendon-driven robotic hand offers the advantage that the motors are set remotely and actuate the hand via tendons that mimic the human musculotendon system. A dynamic model is necessary to control such a complex multi DoFs system. Additionally, having a model enables us to simulate the hand functions with the tendon forces as control input and validate the controller hypothesis.
As the hand group in MIRMI, we are interested in tendon-driven robotic hand design, modeling, control and planning in order to explore and improve the hand manipulations.
Tasks:
- Implementing a dynamic model of the hand prototype using a developing MATLAB modelling toolbox
- Further development and summarizing of the modelling toolbox
- Model parameter identification with experiments
- Dynamic simulation using the generated model
Prerequisites:
- Knowledge of robot kinematic and dynamic modelling
- Basic knowledge of impedance control
- Experience with MATLAB/Simulink programming
- Basic understanding of hand manipulation
- Willing to explore new knowledge domain
If you are interested in this topic and gain more hands-on experience with modelling, please do not hesitate to contact us.
Contact:
Supervisor: M.Sc. Junnan Li (junnan.li(at)tum.de)
Contacting best before 30.09.2024
You can find more research opportunities at the Chairs of our Principal Investigators!
Data Protection Information
When you apply for a position with the Technical University of Munich (TUM), you are submitting personal information. With regard to personal information, please take note of the Datenschutzhinweise gemäß Art. 13 Datenschutz-Grundverordnung (DSGVO) zur Erhebung und Verarbeitung von personenbezogenen Daten im Rahmen Ihrer Bewerbung. (data protection information on collecting and processing personal data contained in your application in accordance with Art. 13 of the General Data Protection Regulation (GDPR)). By submitting your application, you confirm that you have acknowledged the above data protection information of TUM.