Zhe Hu

I am a researcher at the University of Electronic Science and Technology of China. I received my PhD from City University of Hong Kong in February 2022 supervised by Prof.Jia Pan. My research field is Robotics based on Artificial Intelligence.

I worked as an intern at Dorabot Inc, RAL (Robotics and Autonomous Driving Lab) at Baidu Research and Robotics X Lab at Tencent during summer break.

Email | CV | CV Chinese

There are two representative works I did during my undergraduate study and summer break.

The Best Engineering Award for Asia-Pacific Robot Contest (ABU Robocon 2015)

We also made a performance for robot vs human champion (Jiong Dong). See the video.

This robot shows the combination of NLP, CV and Robotics.

I'm interested in any field in Robotics and Cross-Modal problems (robotics, computer vison and natural language processing).

we present a reinforcement-learning (RL)-based algorithm to solve the living object grasping problem. We encode the adversarial behaviors of the living objects into a reward function and train an agent to compete with them.

We present a magnetic tactile sensor for robotic grasping and manipulation. The proposed sensor can measure the tangential force.

we introduce a fuzzy reinforcement learning-based admittance controller that can infer humans' intentions not only through physical interaction but also through natural language. During training, the natural language is encoded into a reward term to help the robot reach the human-intended convergence point, allowing us to develop a ``personalized" policy.

we present a novel architecture of the role adaptation for human-robot physical collaboration. We propose a MDDPG algorithm to estimate the belief of each goal based on the dynamical systems.

A novel texture recognition method is proposed by designing an arc-shaped soft tactile sensor and a bidirectional long short-term memory (LSTM) model with the attention mechanism. By using the proposed method, a respective recognition accuracy of 97% for Braille characters and 99% for 60 types of fabrics have been achieved, revealing the effectiveness of our method in surface texture recognition.

This paper presents a computational framework for automatic optimal robot hand design based on reinforcement learning (RL), which considers desired grasping tasks, grasp control strategies, and performance quality measures altogether.

Living objects are hard to grasp because they can actively dodge and struggle by writhing or deforming while or even prior to being contacted and modeling or predicting their responses to grasping is extremely difficult. This letter presents an algorithm based on reinforcement learning (RL) to attack this challenging problem.

Due to the difficulty of modeling human limb, it is very challenging to design the controller for human-robot collaboration. In this paper, we present a novel controller combining the variable admittance control and assistant control. In particular, the reinforcement learning is used to obtain the optimal damping value of the admittance controller by minimizing the reward function.

n human-robot collaboration, it is crucial for the robot to make its intentions clear and predictable to the human partners. Inspired by the mutual learning and adaptation of human partners, we suggest an actor-critic approach for a legible robot motion planner.

Using DNN to servo-control the shape and position of deformable objects. We also present a robust occlussion removing algorithm in this paper.

In the pipeline of garment manufacturing, the assembly of cloth pieces using fixtures is a widely used technique to reduce the reliance on skilled workers and to improve the sewing quality. In this paper, we present a general visual-based approach to automatically align cloth pieces with target pins without any prior knowledge.

In this letter, we present a robotic navigation algorithm with natural language interfaces that enables a robot to safely walk through a changing environment with moving persons by following human instructions such as “go to the restaurant and keep away from people.”

We present a novel approach for robust manipulation of high-DOF deformable objects such as cloth. Our approach uses a random forest-based controller that maps the observed visual features of the cloth to an optimal control action of the manipulator.

In this letter, we present a general approach to automatically visual servo control the position and shape of a deformable object whose deformation parameters are unknown. The servo control is achieved by online learning a model mapping between the robotic end-effector's movement and the object's deformation measurement. The model is learned using the Gaussian process regression (GPR) to deal with its highly nonlinear property, and once learned, the model is used for predicting the required control at each time step.

The complex physical properties of highly deformable materials such as clothes pose significant challenges for autonomous robotic manipulation systems. We present a novel visual feedback dictionary-based method for manipulating deformable objects towards a desired configuration.

We consider the problem of grasping concave objects, i.e., objects whose surface includes regions with negative curvature. When a multifingered hand is used to restrain these objects, these areas can be advantageously used to determine grasps capable of more robustly resisting to external disturbance wrenches. We propose a new grasp quality metric specifically suited for this case, and we use it to inform a grasp planner searching the space of possible grasps.

Calibrating ADCs in the absence of ground truth presents a significant challenge for high-precision applications. This paper addresses this issue by introducing a novel two-step approach that combines evolutionary strategy and deep learning techniques. First, we employ covariance matrix adaptation evolution strategy to obtain ground truth signal samples with optimal SFDR values. This serves as a robust foundation for the subsequent calibration process. Second, we propose a new calibration neural network architecture called controlled residual convolutional neural networks.

This paper proposes a new digital background calibration technique for interstage gain error (IGE) and gain nonlinearity of pipelined Analog-to-Digital Converters (ADCs). By making the Multiplying Digital-to-Analog Converter (MDAC) work in two modes randomly, two interstage residue curves are obtained. The distance and the geometric relationship between the two residue curves are used to calibrate the IGE and the third-order gain nonlinearity, respectively. In the proposed calibration scheme, the analog circuits need no modification except several additional multiplexers and switches. The merits of this technique include algorithmic simplicity, fast convergence speed, and low power. Simulation results show that, the signal to noise and distortion ratio (SNDR) and the spurious-free dynamic range (SFDR) of a 14-bit 1Gsps pipelined ADC from 44.86 dB and 55.54 dB to 77.99 dB and 86.16 dB after calibration. During the calibration process, the gain nonlinearity and IGE are converged after 2×105 and 2.5×105 sampling cycle, respectively.

Enabling robotic agents to perform complex long-horizon tasks has been a long-standing goal in robotics and artificial intelligence (AI). Despite the potential shown by large language models (LLMs), their planning capabilities remain limited to short-horizon tasks, unable to replace the symbolic planning approach. Symbolic planners, on the other hand, may encounter execution errors due to their common assumption of complete domain knowledge which is hard to manually prepare for an open-world setting. In this paper, we introduce a Language-Augmented Symbolic Planner (LASP) that integrates pre-trained LLMs to enable conventional symbolic planners to operate in an open-world environment where only incomplete knowledge of action preconditions, objects, and properties is initially available. In case of execution errors, LASP can utilize the integrated LLM module to diagnose the cause of the error based on the observation and interact with the environment to incrementally build up its knowledge base necessary for accomplishing the given tasks. Experiments demonstrate that LASP is proficient in solving long-horizon planning problems in the open-world setting, performing well even in situations where there are multiple gaps in the knowledge.