Student Research Programs
Programs
>
Student Research Programs
>
Graduate-level Research in Industrial Projects for Students (G-RIPS) – Sendai 2021
The Industrial Partners for 2021 include:
Project 1: Mitsubishi Electric Corporation: Project A
Title: Development of a Mapping Space for Intuitive Teleoperation with Heterogeneous Devices of Multiple Types
Industrial Partner: Advanced Technology R&D Center of Mitsubishi Electric Corp.
Mitsubishi Electric Corp., founded in 1921, is an electronic and electric equipment manufacturer developing products and solutions in widely diverse fields, including home appliances, industrial equipment, and space technologies. The Advanced Technology R&D Center was established to support the business of Mitsubishi Electric Group through the development of a broad scope of projects covering both basic and new advanced technologies. The main research themes include power electronics, mechatronics, satellite communications, next generation key devices, system solutions for electric power, transportation, factory automation, and automobiles.
Industry Mentors:
Tomoki Emmei, Ph.D., Mitsubishi Electric Corp.
Masaki Haruna, Mitsubishi Electric Corp.
Munetaka Kashiwa, Ph.D., Mitsubishi Electric Corp.
Introduction:
Tele-operation technologies are attracting attention as solutions to reduce labor shortages and work tasks that must be conducted in dangerous areas. In addition to direct applications such as “remote surgical robot that can benefit from the skills of a great doctor anywhere in the world”[1], other applications are emerging recently through studies such as “learning the operator’s actions to promote robot autonomy” [2].
Development of operator-to-edge device mapping represents an important issue for teleoperation technology. Taking robot hand operation as an example, the lengths of operators’ fingers and the sizes of their palms present individual differences. It is therefore necessary to select appropriate mapping of the robot hand in response to changes of operators. Mapping for non-anthropomorphic devices such as a magic hand is not readily apparent compared to the case using anthropomorphic five-fingered robot hands as edge devices. The goal of this project is to construct an appropriate mapping method that enables connection between different structures and multiple types of devices by abstracting the correspondence between these non-anthropomorphic devices.
Technical Background:
Let us take the remote control of a robot hand as an example again. Two typical mapping methods exist for robot hands: joint-2-joint and point-2-point. The joint-2-joint is the method of projecting the joint angle of the operator’s fingers directly onto each joint angle of the robot hand. Although this method is good at “grabbing” objects, it is unsuitable for precise motions because of the difference of fingertip positions between the operator and the robot hand. The point-2-point mapping method, by contrast, overcomes this difficulty by mapping the fingertip positions of the operator to those of the robot. Consequently, the selection of mapping method has a considerably strong effect on the operability and discomfort of the remote control. An appropriate method must be selected depending on the task.
As described above, mapping between devices that have identical geometry is easy. However, proper application of a mapping method between devices with different geometries is not easy. For example, when a three-fingered hand is used as an edge device, movements between the operator and the robot hand do not always correspond intuitively.
Santello et al. proposed an interesting and effective approach to resolve these difficulties [3]. First, the operator’s motion is extracted through three physical quantities: the degree of finger separation (α), the object size (ε), and the degree of finger bending (σ). Then the motion is projected onto the subspace spanned by these parameters. After this subspace is associated with the motion of the robot hand, the operator’s motion is transmitted to the edge device. Using this process, the correspondence between the robot hand and the edge devices is made intuitive and easy by passing the essence of the motions to the subspace from which they are extracted. However, Santello et al. [3] specified that these bases were chosen “on intuition”; some room remains for improvement in the choice of the subspace.
Expectations:
The goal of this project is consideration of a method for selecting a subspace that is intuitive and capable of handling multiple types of edge devices with different structures, referring to an earlier study [3]. Although an inductive approach based on statistics is also possible, as shown in [4], we would like to consider a deductive approach based on knowledge of dynamical systems and nonlinear mapping. Although an earlier report [3] described mappings between heteromorphic hands, the subject matter of this project is not necessarily limited to mapping robotic hands. Because we possess several devices, including robot hands, we intend to conduct experiments with them and to specify the edge devices after selection.
Software Packages and Special Requirements
In addition to an interest in human cognition, movement, and psychology, knowledge in any of the following areas is desirable.
Additionally, we plan to use C++ to implement the robot hand teleoperation. Knowledge and experience in this area are welcome, but are not necessary.
Recommended Reading and References
[1] Enayati, N., De Momi, E., & Ferrigno, G. (2016). Haptics in robot-assisted surgery: Challenges and benefits. IEEE Reviews in Biomedical Engineering, 9(March), 49–65. https://doi.org/10.1109/RBME.2016.2538080
[2] Adachi, T., Fujimoto, K., Sakaino, S., & Tsuji, T. (2018). Imitation learning for object manipulation based on position/force information using bilateral control. 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 3648–3653.
[3] Santello, M., Flanders, M., & Soechting, J. F. (1998). Postural hand synergies for tool use. Journal of Neuroscience, 18(23), 10105–10115. https://doi.org/10.1523/jneurosci.18-23-10105.1998
[4] Meeker, C., Rasmussen, T., & Ciocarlie, M. (2018). Intuitive Hand Teleoperation by Novice Operators Using a Continuous Teleoperation Subspace. IEEE International Conference on Robotics and Automation, 5821–5827. https://doi.org/10.1109/ICRA.2018.8460506
Project 2: Mitsubishi Electric Corporation: Project B
Title: Optimization of wireless base station placement as an essential foundation for our future IoT society
Industrial Partner: Information Technology R&D Center of Mitsubishi Electric Corp.
Mitsubishi Electric is a world leader in the manufacture and sales of electrical and electronic products and systems used in a broad range of fields and applications. As a global leader among green companies, our technologies are being applied to contribute to and support society and daily life around the world. The Information Technology R&D Center is actively creating new businesses through basic research and development in the fields of information technology, media intelligence, electro-optics microwaves, and communication technologies. We are also seeking technologies that reinforce our position on the leading edge of progress, with work to renew existing businesses through the fruits of our R&D in the field of IT.
Industry Mentors:
Takahiro Hashimoto, Mitsubishi Electric Corp.
Kenya Shimizu, Mitsubishi Electric Corp.
Introduction:
Continuous development of wireless communication technologies has led to creation of various IoT applications [1]. Among them, 5G technology [2] is the most promising. The generation of a mobile communication standard has been upgraded almost every decade. Now, the 5G wireless communication standard has been launched with numerous attractive features: enhanced mobile broadband (eMBB), massive machine type communications (mMTC), and ultra-reliable and low latency communications (URLLC). These features will establish 5G technology as a firm foundation for future IoT society.
For 5G, the radio frequency band is to be licensed not only to mobile operators but also to private system developers such as manufacturers. That transformation will alter the industry structure, creating a wide range of business opportunities in automobiles, industrial equipment, home security, and smart meters. That is one reason why we are keenly developing wireless technology.
Construction of an efficient system demands the smooth and rapid development and deployment of new wireless systems. The basic requirements of wireless systems are fewer base stations, complete coverage, and broad band throughput. Most importantly, to use limited radio frequency resources efficiently, such systems must not interfere with other systems. Accurate placement design is important, but the coverage evaluation of a real environment requires great costs and time.
Expectations:
The purpose of the project is creation of an efficient and reliable optimization algorithm to ascertain the positions and directions of wireless base stations to reduce the costs and time for designing wireless systems. The optimization algorithm accepts the spatial distribution of radio signal strength (RSS) to evaluate the cost function. Therefore, we provide a propagation model that can predict the RSS. Participants can change the propagation model input parameters such as the base station position, direction, transmitting power, and the design environment geometry. Then, the propagation model predicts RSS considering the transmission loss which occurs when radio waves pass through walls and when multiple reflections of radio waves occur on the wall.
The design dimensions largely determine the problem difficulty. Base stations are restricted in a one-dimensional space in wireless train control systems [3,4], whereas the positions are in two- or three-dimensional space for general placement design [5]. Typically, the target number of base stations is a few hundred for train control systems, or a few dozen for general placement design. Because of the curse of dimensionality, solving optimization problems becomes extremely difficult as the parameter space increases. We encourage participants to create an algorithm for two- or three-dimensional design space, which is more challenging and worth trying in terms of larger parameter space. For example, the resulting method can be applicable to wireless design for large indoor areas (e.g. factories) or for outdoor areas (e.g. urban environments).
Conventional methods have typically used combinational optimization. In recent years, several studies have examined analytical methods using topology (e.g. persistent homology) [6] or graph theory [7]. We encourage participants to propose formulation using these mathematical structures of coverage and interference and efficient optimization algorithms that are applicable to larger parameter spaces.
Requirements
We welcome students who are motivated to tackle application-oriented and practical problems encountered by manufacturers. Students must have knowledge of optimization. In addition, it is desirable that they have the following knowledge.
References
[1] Internet of Things (IoT), https://internetofthingsagenda.techtarget.com/definition/Internet-of-Things-IoT.
[2] Mitsubishi Electric Begins Demonstrating Local 5G System in Japan, https://emea.mitsubishielectric.com/en/news/releases/global/2020/0518-a/index.html, May 18, 2020.
[3] Wikipedia, Communication-based train control
https://en.wikipedia.org/wiki/Communications-based_train_control
[4] N. Sood, et al., “Integrating Physics-Based Wireless Propagation Models and Network Protocol Design for Train Communication Systems,” in IEEE Transactions on Antennas and Propagation, vol. 66, no. 12, pp. 6635-6645, Dec. 2018.
[5] E. Arribas, et al., “Coverage Optimization with a Dynamic Network of Drone Relays,” in IEEE Transactions on Mobile Computing, vol. 19, no. 10, pp. 2278-2298, 1 Oct. 2020.
[6] Coverage in sensor networks via persistent homology, https://www2.math.upenn.edu/~ghrist/preprints/persistent.pdf
[7] M. Lin, Q. Ye and Y. Ye, “Graph theory based mobile network insight analysis framework,” 2016 IEEE 7th Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON), New York, NY, 2016, pp. 1-7.
Project 3: NEC Corporation
Title: Annealing machine application to artificial neural networks
Industrial Partner: NEC Corporation
NEC (https://www.nec.com), founded in 1899, is now particularly addressing the development of solutions for society that will help resolve many issues the world is facing and which will engender the creation of a brighter and more prosperous society. Through co-creation initiatives with many different stakeholders, including customers, business partners, private individuals, government agencies, and international institutions, we are actively devising new business models to create social value by harnessing our extensive information and communication technology (ICT) assets.
Industryial Mentor: Ryoji Miyazaki, Ph.D. NEC Corp.
Background:
Machine learning is applied increasingly to various industrial problems including image recognition, natural language processing, and material informatics. A recent breakthrough in this field has been led by deep learning based on many-layered artificial neural networks. One such architecture is a deep belief network [1] that can be interpreted as a composition of Boltzmann machines (BMs). A BM is a network which learns the distribution of data [2]. Actually, BMs are named after the Boltzmann distribution, which BMs assume as a data distribution with latent variables to follow. The distribution was introduced in physics to describe thermal equilibrium. The trained network is expected to generate possible “data’’ distributed similarly to the true data. In training, we adjust weights and biases of the network. We must collect statistics based on a model for comparison with those of the true data. Unfortunately, the sampling process requires huge computational costs. Some algorithms based on a rough approximation have been used to avoid computation time growth. For our project, we attempt the application of another technology, an annealing machine, to this problem of sampling.
Great interest has arisen in developing so-called annealing machines over the last decade because combinatorial optimization, the primary target of the annealing machines, is a difficult, ubiquitous problem among many industries. Annealing machines use some algorithms or heuristics such as simulated annealing (SA), which imitates the gradual decrease of the value of a parameter, called a temperature, to find solutions [3]. Actually, SA moves between states for the problems so that the state distribution converges to the Boltzmann distribution at the temperature. In fact, SA “anneals’’ the model to use the fact that the Boltzmann distribution at low temperatures gives the state which minimizes the cost function for a problem with high probability. A task of generating the Boltzmann distribution lies behind SA. It is noteworthy that we face the same task in training BMs. This fact implies that annealing machines can be used to accelerate the BM training process.
NEC has been developing a system for an SA-based algorithm that works on SX-Aurora TSUBASA [4]. SX-Aurora TSUBASA, the latest model of NEC’s supercomputers, has a vector processor that features extremely high memory bandwidth (1.2 TB/s) and 2.45 TFLOPS performance. NEC has implemented software running a variant of SA that takes advantage of the potential of SX-Aurora TSUBASA. This annealing machine, the system composed of SX-Aurora TSUBASA and the software, works well for its intended task of solving combinatorial optimization problems rapidly. However, we are interested in another application. As described above, annealing machines might help to train BMs. Their application to sampling is a frontier of the study of annealing machines. Although they have been investigated extensively, the study of them has been limited to small BMs. The development of large annealing machines, such as NEC’s supported attempts to apply them for large BMs, has continued, but no report of the literature has described their successful application. If applied successfully, then the application might present quite new alternatives to tackle the notoriously difficult problem of sampling in machine learning.
Project and Expectations:
The project is expected to achieve the first application of a large annealing machine, NEC’s system composed of SX-Aurora TSUBASA and software for an annealing algorithm, to the neural network training process, in particular BMs. If their application is successful, then we expect students to demonstrate the benefits and shortcomings of this approach and how it might accelerate standard approaches. Students must develop an algorithm to run BMs using the potential of NEC’s annealing machine. Students will have a unique experience because the system has not been released and because its hardware, SX-Aurora TSUBASA, is a very high performance vector super computer.
Students need not know details of BMs or annealing machines. They can learn topics during this program. Linux experience and Python/C coding skills are necessary. Your skills will be needed to run the annealing machine.
NEC will provide computational environments to students so that they can use the annealing machine on SX Aurora TSUBASA during this program. An example of training datasets will be MNIST [5]: a publicly available database of handwritten digits. Trained BMs can generate artificial images of handwritten digits and can also reconstruct corrupted images. However, we do not restrict students to sole use of MNIST. It would be interesting to use other datasets and to compare the obtained results to ascertain the structures for which the annealing machine shows good/poor learning performance.
Requirements
Linux experience and programming skills (Python and C).
References
[1] G. E. Hinton, S. Osindero, Y.-W. Teh, “A Fast Learning Algorithm for Deep Belief Nets,” Neural Computation 18, 1527 (2006).
[2] G. E. Hinton, “A Practical Guide to Training Restricted Boltzmann Machines,” in Neural Networks: Tricks of the Trade, Second ed., Lecture Notes in Computer Science, edited by G. Montavon, G. B. Orr, K.-R. Miller (Springer, New York, 2012), 7700, 599.
[3] S. Kirkpatrick, C. D. Gelatt Jr., M. P. Vecchi, “Optimization by Simulated Annealing,” Science 220, 671 (1983).
[4] NEC’s website on SX-Aurora TSUBASA, https://www.hpc.nec/
[5] Y. LeCun, C. Cortes, C.J. C. Burges (1998) The MNIST database of handwritten digits. http://yann.lecun.com/exdb/mnist/
Project 4: F-MIRAI
Title: Design for next-generation mobility service in suburban areas
Industrial Partner: F-MIRAI center at University of Tsukuba
Toyota will lead the way to the future of mobility, enriching lives worldwide with the safest and most responsible ways of moving people. In the near future, cars are expected to connect with people and communities and to perform in new roles as part of human social infrastructure. New domains of service such as AI, autonomous driving, robotics, and connected cars are becoming especially important. Toyota aims to reach the ultimate goal of sustainable mobility, creating a mobile future society full of smiles. Toyota and the University of Tsukuba have jointly established the R&D center for strategic frontiers in social planning (F-MIRAI), which advances toward Society 5.0 through the development of infrastructure for future communities and the formation of industrial centers through long-term collaborative action.
Industrial mentor: Hiroyasu Ando, Ph. D., F-MIRAI, University of Tsukuba, and Center of Mathematical Science for Open Innovation in AIMR, Tohoku University
Background:
Rapid suburban sprawl in metropolitan areas in Japan has led to the rapid development of peripheral satellite cities. High-speed railway networks and motorways have contributed considerably to improving connections among established metropolitan areas and these new peripheral cities. Nevertheless, rapid population growth has led simultaneously to construction of suburban sprawls without any schematic road planning.
Consequently, key functions of cities for the daily life of residents, such as commerce, administration, and schools, have diffused to suburban areas on the basis of widening use of private cars. Such development is neither sustainable nor desirable as populations age and as energy conservation becomes increasingly important. In principle, a higher quality of life can be achieved if key city functions are aggregated in city centers around main railway stations. This aggregation necessitates the adoption of better and more efficient public transportation networks in conjunction with small devices used for personal mobility.
Mobility-as-a-Service (MaaS*) describes the integration of different transportation modes to achieve mobility solutions that are consumed as a service. Mobility innovation can be implemented fully or partially as MaaS and Smart Cities. Its principles are often designated as the following: C, Connected; A, Autonomous; S, Sharing; E, Electric (CASE). A wider range of demonstration programs and commercial services is being provided to improve transportation safety, efficiency, and convenience. In particular, “IoT vehicle information” is anticipated for application to widely various services because it aggregates various information related to local transportation in an immediate, remote, and wide-area manner. The benefits of using IoT vehicle information with AI to meet transportation challenges and support local economic growth are important.
* MaaS: integration of various forms of transport services into a single mobility service that is accessible on demand. To meet a customer’s request, a MaaS operator provides a diverse menu of transport options, be they public transport, ride-, car- or bike-sharing, taxi or car rental/lease, or some combination thereof. For the user, MaaS can offer added value through use of a single application to provide access to mobility, with a single payment channel instead of multiple ticketing and payment operations. For its users, MaaS is expected to be the best value proposition, helping users meet their mobility needs and solve the inconvenient aspects of individual journeys as well as the entire system of mobility services.
Source: MaaS Alliance, https://maas-alliance.eu/homepage/what-is-maas/
* Smart city in Tsukuba: Tsukuba City is working on a “smart city” initiative aimed at solving problems in widely diverse areas such as administrative services, transportation, medical and nursing care, and infrastructure, using advanced technologies such as ICT and other data to create a sustainable citizen-centered city in which everyone can live safely, conveniently, and comfortably.
Project and Expectations:
Many studies and demonstration experiments have elucidated MaaS as an implementation of CASE, particularly for metropolitan areas where other multiple transportation modes such as rails, metro, and buses are available. Nevertheless, other areas for which private cars constitute the dominant transportation mode have not been investigated thoroughly. Mobility services for non-metropolitan areas can differ from those for metropolitan areas because of differences in population density/distribution, transportation demand, and public transportation networks. As an example, Tsukuba city, a city with a population of 241,000 people, and located 60 km northeast of Tokyo, has a transportation system that is dependent on private cars. However, Tsukuba city was designed as a science city such that the University of Tsukuba, which also has a hospital, is positioned as the center of its urban function. Therefore, MaaS centered on the university can be a good example of a future transportation system in a non-metropolitan area.
This project will design MaaS for practical cases for the campus of University of Tsukuba through analyses of person-trips and other related datasets. The present major transportation modes around the university are private cars, buses, bicycles, and walks. The services accompanied with mobility around the campus are various, ranging from education, health care, and nursing care, to childcare. Participants are expected to estimate the overall transportation demand to optimize those services through analyses of available datasets, and to build a mathematical model as a system-of-systems based on some of the following perspectives. All can be considered with automated vehicles.
Although several approaches for such problems already exist, participants are expected to find completely new mathematical approaches, descriptors, formulations, solvers, visualizations, operation plans, etc. to accommodate and support our future society with highly sophisticated modes of mobility. Additionally, investigating the energy expenditure and traffic flow of cars in terms of dynamical systems theory is possible by implementing mathematical models into automated vehicles in a miniature traffic course on a trial basis. (See video in Ref. [9].)
Requirements
We welcome applications from motivated team-players who have knowledge and practical skills related to one or more of the following.
Recommended Readings and References
[1] Applied Mathematical Programming. http://web.mit.edu/15.053/www/AMP.htm
[2] Nonlinear Dynamics and Chaos. https://www.youtube.com/playlist?list=PLbN57C5Zdl6j_qJA-pARJnKsmROzPnO9V
[3] Toyota Motor Corporation. Toyota e-Palette Concept Opening Video. 2018. https://youtu.be/bniK9Eqgnw4
[4] Toyota Motor Corporation. Toyota e-Palette Concept Basic Function Video. 2018. https://youtu.be/7nhY0eHUUEo
[5] OECD/International Transport Forum. Shared Mobility: Innovation for Liveable City. 2016. https://www.itf-oecd.org/itf-work-shared-mobility
[6] G-RIPS Sendai 2018. Toyota Group Final report: Data-Driven Models for Predictive Control of Toyota’s e-Palette Mobility System. 2018. (For those who have decided to participate)
[7] G-RIPS Sendai 2019. Toyota Group Final report: Implementation of TOYOTA’s e-Palette Mobility System to Develop Data-Driven Optimization of Transportation Networks. 2019. (For those who have decided to participate)
[8] Mathematical Challenges and Opportunities for Autonomous Vehicles Tutorials IPAM long program: http://www.ipam.ucla.edu/programs/workshops/mathematical-challenges-and-opportunities-for-autonomous-vehicles-tutorials/?tab=schedule
[9] Miniature traffic model by F-MIRAI in University of Tsukuba (in Japanese): https://youtu.be/NvHfrb0K6Fo
Copyright. All Rights Reserved.