(INCORP W/RIPS LA 2020) Graduate-level Research in Industrial Projects for Students (G-RIPS) – Sendai 2020

June 15 - August 7, 2020

Sponsors and Projects

The sponsors for 2020 include:

 

Project 1: TOYOTA Motor Corporation

Title: Design for the next generation mobility service in suburban areas

Sponsor: Toyota Motor Corporation and University of Tsukuba

Toyota will lead the way to the future of mobility, enriching lives worldwide with the safest and most responsible ways of moving people. In the near future, cars are expected to connect with people and communities and to take on new roles as part of the social infrastructure. New areas, such as AI, autonomous driving, robotics and connected cars, are becoming especially important. Toyota aims to reach the ultimate goal of sustainable mobility, creating a mobile future society full of smiles. Toyota and the University of Tsukuba have jointly established the R&D center for strategic frontiers in social planning which advances toward Society 5.0 through the development of infrastructure for future communities and the formation of industrial centers through long-term collaborative action.

Background:

Rapid suburban sprawl in metropolitan areas in Japan has led to the rapid development of peripheral satellite cities. High-speed railway networks and motorways have contributed considerably to improving connections among established metropolitan areas and these new peripheral cities. However, simultaneously, because of rapid population growth, sprawled suburban areas with small-scale residential developments have been constructed without schematic road network planning.

Consequently, key functions of cities for residents’ lives, such as commerce, administration, and schools, have diffused to suburban areas on the premise of widening use of private cars. Nevertheless, such developments are not sustainable or desirable as the population ages and as energy conservation becomes more important. In principle, a higher quality of life can be achieved if key city functions are aggregated in city centers around main railway stations. This aggregation necessitates better and more efficient public transportation networks in conjunction with small devices for personal mobility.

Mobility-as-a-Service (MaaS*) describes the integration of different transportation modes to achieve mobility solutions that are consumed as a service. Mobility innovation is able to be implemented fully or partially as MaaS. It is often designated as the following: C, Connected; A, Autonomous; S, Sharing; E, Electric (CASE). A wider range of demonstration programs and commercial services is being provided to improve transportation safety, efficiency, and convenience.

* MaaS: is the integration of various forms of transport services into a single mobility service accessible on demand. To meet a customer’s request, a MaaS operator facilitates a diverse menu of transport options, be they public transport, ride-, car- or bike-sharing, taxi or car rental/lease, or a combination thereof. For the user, MaaS can offer added value through use of a single application to provide access to mobility, with a single payment channel instead of multiple ticketing and payment operations. For its users, MaaS should be the best value proposition, by helping them meet their mobility needs and solve the inconvenient parts of individual journeys as well as the entire system of mobility services.

Source: MaaS Alliance, https://maas-alliance.eu/homepage/what-is-maas/

Project and Expectations:

Many studies and demonstration experiments have elucidated MaaS as an implementation of CASE, particularly for metropolitan areas where other multiple transportation modes such as rails, metro, and buses are available. Nevertheless, other areas where private cars constitute the dominant transportation mode are not investigated thoroughly. Mobility services for non-metropolitan areas can be different from those for metropolitan areas because of differences in population density/distribution, transportation demand, and public transportation networks.

As an example, Tsukuba city, a city with a population of 241,000 people, and located 60 km northeast of Tokyo, has a transportation system that is dependent on private cars. This project will design MaaS for practical cases, particularly for university campuses, through analyses of person-trips and other related datasets. The University of Tsukuba has a large campus area and numerous students. The present major transportation modes are buses, bicycles and walks. Participants are expected to estimate the overall transportation demand through analysis of available datasets, and to then build a practical and unique model to optimize MaaS for the University of Tsukuba campus.

Although several approaches for such problems already exist, students are expected to find completely new mathematical approaches, descriptors, formulations, solvers, visualizations, operation plans, etc. to accommodate and support our future society with highly sophisticated modes of mobility. This project includes site visits to a mobility exhibition center in Tokyo and the University of Tsukuba.

Requirements

We welcome applications from motivated team-players who have knowledge and practical skills related to one or more of the following.

  • Mathematical statistics
  • Optimization
  • Machine/Deep learning
  • Operations research
  • Programing language (Python, C, or MATLAB)

Recommended Readings and References

[1] Applied Mathematical Programming. http://web.mit.edu/15.053/www/AMP.htm

[2] Model Building in Mathematical Programming, Fifth Edition. https://www.wiley.com/en-us/Model+Building+in+Mathematical+Programming%2C+5th+Edition-p-9781118443330

[3] Toyota Motor Corporation. Toyota e-Palette Concept Opening Video. 2018. https://youtu.be/bniK9Eqgnw4

[4] Toyota Motor Corporation. Toyota e-Palette Concept Basic Function Video. 2018. https://youtu.be/7nhY0eHUUEo

[5] OECD/International Transport Forum. Shared Mobility: Innovation for Liveable City. 2016. https://www.itf-oecd.org/itf-work-shared-mobility

[6] GRIPS-Sendai 2018. Toyota Group Final report: Data-Driven Models for Predictive Control of Toyota’s e-Palette Mobility System. 2018. (to be provided later)

[7] g-RIPS Sendai 2019. Toyota Group Final report: Implementation of TOYOTA’s e-Palette Mobility System to Develop Data-Driven Optimization of Transportation Networks. 2019. (to be provided later)

 

Project 2: FUJITSU Laboratories Ltd 

Title: Resolving real-world issues by “Digital Annealer”

Sponsor: Fujitsu Laboratories Ltd.

Fujitsu Laboratories, established in 1968,is an organization independent from Fujitsu that conducts the world’s top-level technological development in a free atmosphere for researchers. Fujitsu is the leading Japanese information and communication technology (ICT) company, offering a full range of technology-related products, solutions, and services. We have produced many innovative results through IT and have been leading the world by providing new value to people, society, and businesses. Fujitsu,number 1 in Japan, is the world’s seventh largest IT service provider.

Industry Mentors:

Masato Wakamura, Ph.D., Fujitsu Laboratories Ltd., Expert

Makoto Nakamura, Ph.D., Fujitsu Laboratories Ltd.

Introduction:

Digital Annealer (DA) http://www.fujitsu.com/global/digitalannealer/is a new technology to solve large-scale combinatorial optimization problems instantly. Using a digital circuit design inspired by quantum phenomena, it can solve problems that are difficult for classical computers to address. Many real-world social issues such as environmental problems can be regarded as combinatorial optimization problems. The objective of this project is to solve a serious societal issue using DA.

Technical Background:

Quantum computing technologies can be categorized into two types: quantum gate computers and Ising machines [1, 2]. Quantum gate computers are designed for universal computing. Ising machines are specialized forseeking solutions to combinatorial optimization problems.

Moreover, Ising machines of two types exist; quantum annealing machines and simulated annealing machines. A quantum annealing machine seeks solutions using quantum bits, which are made of quantum devices such as superconducting circuits. A simulated annealing machine uses a digital circuit. In fact, DA is a type of simulated annealing machine with a new digital circuit architecture that is designed to solve combinatorial optimization problems efficiently. The number of bits (node) of an annealing machine is related closely to the number of combination parameters. Annealing machines with numerous bits (nodes) can solve large-scale combinatorial optimization problems.

Many of the world’s social issues can be treated as combinatorial optimization problems that cannot be easily solved using conventional computers when the problem size increases. The applicable fields include chemistry, finance, and transportation. To solve an Ising model [6–9] to which a combinatorial optimization problem has been converted, DA uses Markov chain Monte Carlo (MCMC) method [3] and Simulated Annealing (SA) [4, 5]. This conversion technique is extremely important to solve problems rapidly.

The general procedure to solve problems using DA is described below.

1)  Problem description

2)  Formulation to a combinatorial optimization problem

3)  Re-formulation to an Ising model

4)  Conversion into quadratic unconstrained binary optimization (QUBO)

5)  Calculation of optimal solution using DA

Fujitsu launched a DA cloud service in May 2018. The first generation has 1024 bits, all of which are fully connected with 16-bit precision. Although some problems can be accommodated within this bit size, many real-world problems require a larger bit scale.

https://www.fujitsu.com/jp/documents/digitalannealer/services/da-shoukaiEN.pdf

Fujitsu has developed a second generation service of DA using Digital Annealer Unit (DAU), which has an 8K bit scale.

https://www.fujitsu.com/global/about/resources/news/press-releases/2018/1221-01.html

After making both hardware and software enhancements, DA can accommodate problems on a scale of 100K bits.

Expectations:

The main aim of this project is the identification of concrete formulations and efficient algorithms fitted to DA and, ultimately,the discovery of optimal solutions for combinatorial optimization problems. Students will obtain experience at thinking about solving social issues in the real world, andwill observe that many real-world problems can be regarded as combinatorial optimization problems. The project specifically examines flow optimization problems among widely diverse combinatorial optimization problems. Examples of flow optimization problems are presented below.

1) Route optimization for mitigating traffic congestion

Reduce overall travel time by assigning different routes throughout the city or throughout an entire factory.

2) Optimization of work flow line in a factory to improve productivity

Up to a 45% reduction in moving distances in a warehouse was achieved using Fujitsu IT Products http://www.fujitsu.com/global/digitalannealer/case-studies/201804-fjit/

3) Multiple Traveling Salesman and Target Visitation Problems (MTSP&TVP)

MTSP and TVP were formulatedby a Fujitsu team in the g-RIPS-Sendai-2019 program. These formulations are applicable to optimization problems in logistics planning to evacuation shelters in severe disasters. (The final report of the Fujitsu team in g-RIPS-Sendai 2019 will be provided to 2020 project members.)

Students can select one of these examples and seek new formulations, or identify new societal problems. Students are expected to examine the problem scale that is calculable with DA and the combination scales of real problems. It is also expected that the students will try to devise a method of obtaining an accurate calculation result. Fujitsu will provide user accounts and computational environments of the newest DA machine to project members during g-RIPS-Sendai 2020, with support from technical team.

Requirements:

Programming (mandatory, Python; optional, C)

Recommended Reading and References:

[1] P. W. Shor, “Algorithms for Quantum Computation: Discrete Logarithms and Factoring,” Proceedings, 35th Annual Symposium on Foundations of Computer Science, Santa Fe, NM, November 20–22, 1994, IEEE Computer Society Press, pp. 124–134.

[2] P. Bunyk et al., “Architectural Considerations in the Design of a Superconducting Quantum Annealing Processor,” IEEE Trans. Applied Superconductivity, vol. 24, no. 4, (2014).

[3] N. Metropolis et al., “Equation of State Calculations by Fast Computing Machines,” J. Chem. Phys.,vol. 21, pp. 1087 (1953).

[4] S. Kirkpatrick et al., “Optimization by Simulated Annealing,” Science vol. 220, no. 4598, pp. 671–680 (1983).

[5] S. Geman and D. Geman, “Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images,” IEEE Trans. on Pattern Analysis and Machine Intelligence, PAMI-6, pp.721–741, 1984.

[6] Hopfield, J.J.,“Neural networks and physical systems with emergent collective computational abilities,” Proc. Natl. Acad. Sci. vol. 79, no. 8, pp. 2254–2558 (1982).

[7] Aarts, Emile H.L., Korst, Jan H.M., Boltzmann machines and their applications. In: Bakker, J.W., Nijman, A.J., Treleaven, P.C. (eds.) PARLE 1987. LNCS, vol. 258, pp. 34–50. Springer, Heidelberg (1987).

[8] Skubiszewski, M.,“An exact hardware implementation of the Boltzmann machine.” In: Proceedings of the Fourth IEEE Symposium on Parallel and Distributed Processing, pp. 107–110 (1992).

[9] Yamaoka, M. et al.,“20 k-spin Ising Chip for Combinational Optimization Problem with CMOS Annealing,” ISSCC 2015 Digest of Technical Papers, pp. 1–3, February 2015.

 

Project 3: NEC Corporation 

Title: Annealing machine application to artificial neural networks

Sponsor: NEC Corporation

NEC, founded in 1899, is now particularly addressing the development of solutions for society that will help resolve many issues the world is facing and which will engender the creation of a brighter and more prosperous society. Through co-creation initiatives with many different stakeholders, including customers, business partners, private individuals, government agencies, and international institutions, we are actively devising new business models to create social value by harnessing our extensive information and communication technology (ICT) assets.

Background:

Machine learning is applied increasingly to various industrial problems including image recognition, natural language processing, and material informatics. A recent breakthrough in this field has been led by deep learning based on many-layered artificial neural networks. One such architecture is a deep belief network [1] that can be interpreted as a composition of Boltzmann machines (BMs). A BM is a network which learns the distribution of data [2]. Actually, BMs are named after the Boltzmann distribution, which BMs assume as a data distribution with latent variables to follow. The distribution was introduced in physics to describe thermal equilibrium. The trained network is expected to generate possible “data’’ distributed similarly to the true data. In training, we adjust weights and biases of the network. We must collect statistics based on a model for comparison with those of the true data. Unfortunately, the sampling process requires huge computational costs. Some algorithms based on a rough approximation have been used to avoid computation time growth. For our project, we attempt the application of another technology, an annealing machine, to this problem of sampling.

Great interest has arisen in developing so-called annealing machines over the last decade because combinatorial optimization, the primary target of the annealing machines, is a difficult, ubiquitous problem among many industries. Annealing machines use some algorithms or heuristics such as simulated annealing (SA), which imitates the gradual decrease of the value of a parameter, called a temperature, to find solutions [3]. Actually, SA moves between states for the problems so that the state distribution converges to the Boltzmann distribution at the temperature. In fact, SA “anneals’’ the model to use the fact that the Boltzmann distribution at low temperatures gives the state which minimizes the cost function for a problem with high probability. A task of generating the Boltzmann distribution lies behind SA. It is noteworthy that we face the same task in training BMs. This fact implies that annealing machines can be used to accelerate the BM training process.

NEC has been developing a system for an SA-based algorithm that works on SX-Aurora TSUBASA [4]. SX-Aurora TSUBASA, the latest model of NEC’s supercomputers, has a vector processor that features extremely high memory bandwidth (1.2 TB/s) and 2.45 TFLOPS performance. NEC has implemented software running a variant of SA that takes advantage of the potential of SX-Aurora TSUBASA. This annealing machine, the system composed of SX-Aurora TSUBASA and the software, works well for its intended task of solving combinatorial optimization problems rapidly. However, we are interested in another application. As described above, annealing machines might help to train BMs. Their application to sampling is a frontier of the study of annealing machines. Although they have been investigated extensively, the study of them has been limited to small BMs. The development of large annealing machines, such as NEC’s supported attempts to apply them for large BMs, has continued, but no report of the literature has described their successful application. If applied successfully, then the application might present quite new alternatives to tackle the notoriously difficult problem of sampling in machine learning.

Project and Expectations:

The project is expected to achieve the first application of a large annealing machine, NEC’s system composed of SX-Aurora TSUBASA and software for an annealing algorithm, to the neural network training process, in particular BMs. If their application is successful, then we expect students to demonstrate the benefits and shortcomings of this approach and how it might accelerate standard approaches. Students must develop an algorithm to run BMs using the potential of NEC’s annealing machine. Students will have a unique experience because the system has not been released and becauseits hardware, SX-Aurora TSUBASA, is a very high performance vector super computer.

Students need not know details of BMs or annealing machines. They can learn topics during this program. Linux experience and Python/C coding skills are necessary. Your skills will be needed to run the annealing machine.

NEC will provide computational environments to students so that they can use the annealing machine on SX Aurora TSUBASA during this program. An example of training datasets will be MNIST [5]: a publicly available database of handwritten digits. Trained BMs can generate artificial images of handwritten digits and can also reconstruct corrupted images. However, we do not restrict students to sole use of MNIST. It would be interesting to use other datasets and to compare the obtained results to ascertain the structures for which the annealing machine shows good/poor learning performance.

Requirements:

Linux experience and programming skills (Python and C).

References:

[1] G. E. Hinton, S. Osindero, Y.-W. Teh, “A Fast Learning Algorithm for Deep Belief Nets,” Neural Computation 18, 1527 (2006).

[2] G. E. Hinton, “A Practical Guide to Training Restricted Boltzmann Machines,” in Neural Networks: Tricks of the Trade, Second ed., Lecture Notes in Computer Science, edited by G. Montavon, G. B. Orr, K.-R. Miller (Springer, New York, 2012), 7700, 599.

[3] S. Kirkpatrick, C. D. Gelatt Jr., M. P. Vecchi, “Optimization by Simulated Annealing,” Science 220, 671 (1983).

[4] NEC’s website on SX-Aurora TSUBASA, https://www.hpc.nec/

[5] Y. LeCun, C. Cortes, C.J. C. Burges (1998) The MNIST database of handwritten digits. http://yann.lecun.com/exdb/mnist/