Hitachi 9900 manual

[...]

[...]

i[...]

Hi tac hi Dat a S y s t e m s ii[...]

T able of Contents T able of Contents ......................................................................................................................................iii T able of Figures ........................................................................................................................................vii T able of T able[...]

A dvanced Cache Algorithms of the Lightning 9900 V Series ....................................................................18 H itachi FlashAccess Software Allows Data Sets to be Permanently Placed in Cache ................................18 Read-ahead for High-performance Sequential Reads ........................................................[...]

Chapter 9 – Hitachi F reedom Storage Software Solutions Are Best of Breed ______________________33 Hitachi F reedom Storage Software Solutions Simplifies Information Management, P rotects Data Assets, and Optimizes Storage Investments ..........................................................................................................33 Stor[...]

Chapter 11 – Optimizing Performance, Availability, and Cost with the H iReturn Investment Analysis T ool Kit ________________________________________49 HiReturn Investment Analysis T ool Kit ....................................................................................................49 The HiReturn Methodology Is Implemented in Three Phase[...]

T able of Figures F igure 1 The Lightning 9980V system pr o vides for exploding gr owth and data hyper-consolidation while the Lightning 9970V system pr ovides for managed gr owth and open systems exploitation. ............................................................................................2 F igure 2 The Lightning 9900 V Series is base[...]

Hi tac hi Dat a S y s t e m s viii T ables T able 1 Comparison of Lightning 9900 V Series models. ..................................................................2 T able 2 B andwidth of supported interfaces of the Lightning 9900 V Series. ....................................16 T able 3 S ystem capacities and number of disk drives for various con[...]

1 I ntroduction Red ef ini ng the In du stry Stan dar d, O nce Ag ain Hitachi Data Systems, in presenting the Hitachi F reedom Storage ™ Lightning 9900 ™ V S eries internal switched storage architecture with virtual Fibre Channel switches, again r edefines the industry standard for “best-of-breed” storage systems. It is the most advanced de[...]

Li ghtn in g 9900 V Ser ie s Pro duc t Of f e r i n g s The Lightning 9900 V Series offers two models – the Lightning 9980V ™ and the Lightning 9970V ™ . The Lightning 9980V system is recommended for users requiring large storage capacities of 149TB+ of raw storage capacity . With the same advanced architectur e as the Lightning 9980V system,[...]

An Over vi ew of the Ligh tni ng 9900 V Ser ies Ar c h i t e c t u re The Lightning 9900 V Series intelligent storage systems are the most po werful enterprise systems in the industry today and two generations ahead of the competition. The new architecture used in the Lightning 9900 V Series has almost twice the internal processing pow er and bandw[...]

The main components that make up the Lightning 9900 V Series systems are the Array Control Pr ocessor modules (ACPs) and Host Adapter modules, called Channel Host Interface Pr ocessors (CHIPs). These elements enable the unsurpassed bandwidth of the Hi- S tar network by bringing their own network paths (and bandwidth) to the Cache Hierarchical Star [...]

• S eparate storage for control and configuration tables • M irrored cache instead of single image cache 2 • O ffered by Hitachi Data Systems, the leading supplier of storage systems for geographically GDPS ™ production sites • N anoCopy ™ technology which offers a scheme for disaster reco ver y • High-availability, custom-designed di[...]

Seco nd -ge ner ati on “V irtual ” Swit ched Stor ag e An important key to lo wer TCO is the Lightning 9900 V Series “virtual” Fibr e Channel ports. V irtualization is becoming increasingly important in today ’ s heterogeneous storage environment, which is made up of these storage architectures: direct attached storage (DAS), storage area[...]

As shown in Figure 6, the heterogeneous solutions of today require the separate configuration of physical ports for each host platform type supported, particularly in high availability path failo v er and clustered environments. This both increases expense through unnecessary extra ports and limits heterogeneous consolidation, resulting in higher T[...]

The Lightning 9900 V Series pr ovides “virtual” Fibre Channel ports that are logically managed by intelligent Fibre Channel controller cards. Each physical Fibre Channel port may have multiple Host Storage Domains. A Host Storage Domain supports a “logical” F ibre Channel port (Fig. 7) each with its own set of LUNs. Hosts are matched to the[...]

9 H itachi F r eedom Storage ™ Lightning 9900 ™ V Series P ackaging Th e Hit achi Fr eed om Sto ra ge Ligh t ning 9 980V ™ Sy ste m Pa ckagi ng The Lightning 9980V system is available in a two-to-five cabinet configuration (as shown on the co v er of this guide), depending on connectivity and capacity needs. A Control Frame serv es as the con[...]

Li ghtn in g 9980 V Syst em Arr ay Frame s Each array frame of the Lightning 9980V system contains up to 256 Hard Disk Drive (HDD) modules and communication interfaces to the Control Frame, for a total of 1024 drives for a maximum configured system. An Array Frame is shown in Figure 9. This design allows for efficient centralized and consolidated s[...]

Cables carry 48-volt main system po w er from the back of the Control Frame and Array F rame where the po w er is converted to the required voltages. Different AC pow er cord options allow all Lightning 9900 V Series models to connect with the different electrical outlet configurations used around the world. Hit ach i Fr ee dom St orag e Lig htn in[...]

Th e Sy stem Moni tori ng Ne two rk of the Lig htnin g 9 900 V Ser ies Lightning 9900 V Series systems are designed with a system-monitoring network. The system-monitoring cables are connected to all cabinets in Lightning 9900 V Series systems, and they report a variety of system parameters, such as component failure, fan speed, po wer , v oltages,[...]

13 F ront-end Design of the H itachi F r eedom Storage ™ Lightning 9900 ™ V Series CH IPs S uppor t Fi bre Chan nel Hos t and ESC ON ® or FICON ™ H o s t Co nne cti vity and Sca labi l ity The Lightning 9900 ™ V Series supports both Fibre Channel for open systems and ESCON ® or FICON ™ for IBM ® S/390 ® - and zOS ™ - compatible plat[...]

Abo ut th e Fibr e Ch ann el Inte rf ace Bo ar d s Each Fibre Channel Interface Board, as shown in Figure 13, has four Interface Pr ocessors for management of connectivity to the host. Each port supports a transfer rate of 2Gbit/sec (gigabits/second) in conformance with the Fibre Channel standard. An optional eight-port Fibre Channel Interface Boar[...]

There are four ports for each interface board or a total of 32 host interfaces for a fully configured system. An ESCON interface board is shown in Figure 14. A FICON interface board has a similar appearance. The FICON interface card is available with 1Gbit/sec or 2Gbit/sec transfer rates. A Lightning 9900 V Series system can connect to both mainfra[...]

Ba ndwi dth of Suppo rt ed Hos t Int erf ace s Th e band width of inte rfa ces su pport ed by the Lig htnin g 9900 V Seri es is sh own in T a ble 2. Supported Interface Bandwidth Fibre Channel Adapters 100MB/sec or 200MB/sec ESCON ® Adapters 17.5MB/sec FICON ™ Adapters 100MB/sec or 200MB/sec Nu mber of Hos t Int erfa ces Supp ort ed U p to 64 Fi[...]

17 H i-Star ™ Architecture – An I nternal Switched H ierarchical Star Networ k Th e In te rnal Hi- Sta r ™ A rc h i t e c t u r e The internal Hi-Star architecture consists of two separate networks: the Cache Hierarchical Star Network (C-HSN) and the Control Memory Hierarchical Star Networ k (CM-HSN). Th e Ca che Hi erar chic al Star Net wo r[...]

Th e Ca che Sw it ch Pr ov ide s Non- bl ocki ng Swit ched Acc ess to Cache A t the heart of the Lightning 9900 V Series are four Cache Switches (CSWs). T ogether, these four switches (in the Lightning 9980V ™ system) use a parallel switch fabric bus (PSFB). The CSW is a specially designed crossbar switch that functions as a combination MUX, path[...]

mirrored images of each other . This is illustrated in Figure 17. Control data is “data about data,” also called “metadata. ” Essentially, control information is handled “ out of band” from the data paths, both through a separate memory area and network. The CM-HSN is a much simpler network design in that every connection is a point-to-[...]

Hi tac hi Dat a S y s t e m s 20[...]

21 H igh-performance Back-end Design Hig h- spee d All Fib r e Cha nne l Back -end Des ign S pecial high-performance back-end drives are available with the Hitachi F reedom Storage ™ Lightning 9900 ™ V Series. All the drives are 3.5" Low P rofile (LP) 1"-height form factor with capacities available in 36GB, 73GB, and 146GB drives. The[...]

Ma ximu m Con fig ur ati on of ACP s and Di sk s The maximum configuration of four ACPs and 16 Fibre Channel loops is shown in F igure 19. New Fi br e Cha nne l Dis k Driv es The back-end of the Lightning 9900 V Series uses dual ported/dual active Fibre Channel disk drives. All the drives are 3.5" Low P r ofile (LP) 1"-height form factor [...]

23 Capacities and RAID Architectur e St orag e Capa c it y of the Hita chi Fr eedo m Stor age ™ Lig htn in g 9900 ™ V Seri es The Lightning 9900 ™ V Series supports an unmatched range of capacities as shown in T able 3. Adv ant ag es of the Li ghtni ng 990 0 V Seri es RAI D Har d w a re H itachi designs the most advanced RAID (Redundant Array[...]

Hi gh- dens it y Dis k Dr ive (H DD) Asse mbli es The disk drives used in the Lightning 9900 V Series are specially designed for high availability and performance. Each drive has the industry unique feature of supporting dual active ports. This pr ovides significantly increased performance. Figure 20 – The Lightning 9900 V Series systems are avai[...]

H itachi F r eedom Storage ™ Lightning 9900 ™ V Series A vailability Is Highest in the I ndustr y An Ov e rv iew o f Hi gh- ava ila bi lit y ( HA) Co mpu t i ng So f tw ar e I t is increasingly important for a business to support continued access to global information 24/7. What’ s more, careers often depend on the availability of service lev[...]

• D ual data paths and dual control paths connecting every component • M irrored cache for all write data instead of single image cache 4 • IBM ® Geographically Dispersed Parallel Sysplex ™ (GDPS ™ ) support • Hitachi T r ueCopy ™ software combined with Hitachi ShadowImage ™ software for disaster recovery • N anoCopy ™ technolo[...]

There are four basic types of high-availability middleware that can reduce downtime in the event of a data path or host failure. The Lightning 9900 V Series supports each classification of middleware: A l t e r n a te P a th i ng M id dl ewa re Sw itc he s the I/O L oa d i n th e E ve nt of Pa t h F a il ur e This type of middleware automatically s[...]

Pa ra lle l Da tab a se Cl ust e ri ng This type of middleware is a special version of host failo ver middleware, which supports major parallel database ser vers like Oracle ® P arallel Ser ver, Informix ® XPS, and Sybase ® MPP. Clustering middleware supports distributed lock management, a feature that enables parallel database software running [...]

29 Hitachi Fr eedom Storage Lightning 9900 V Series A v ailability is Highest in the Industry S t a n d a r d Re d un d an t Fan s fo r Co o li ng The Lightning 9900 V Series pr o vides fully redundant fans for uninterrupted cooling to all components of the system. The speed and operation of these fans are monitored by the environmental monitoring [...]

Hitachi Data Systems 30[...]

31 U nmatched Performance and Scalability T ruly Sca labl e Pe rform a nce Based on the internal switched Hi-Star ™ architecture, the Hitachi Freedom Storage ™ Lightning 9900 ™ V Series shows unmatched performance scalability. This chapter discusses the ability to dramatically add more online transaction processing (OLTP) users and mixed deci[...]

ov ershot the targeted transfer rate for the DSS system by appr o ximately 50MB/sec. This was managed by using Hitachi Priority Access software to limit the throughput for a particular application. At this point, the two OLTP workloads were still able to sustain their 2200TPM each, or 4400TPM aggregate. The three combined workloads were allo wed to[...]

H itachi F r eedom Storage ™ S oftware Solutions Ar e Best-of-B r eed Customers and analysts regard the Hitachi F reedom Storage ™ Lightning 9900 ™ V S eries as the most advanced products in the enterprise storage system market. The Hitachi F reedom Storage Software Solutions Guide (Figure 24) discusses the advantages of the suite of leading-[...]

Hi tac hi Dat a S y s t e m s 34 Storage Area Management Suite HiCommand ™ Management Framework ✔ ✔ ✔ ✔ - HiCommand Device Manager ✔ ✔ ✔ ✔ - HiCommand Tuning Manager ✔ ✔ ✔ ✔ Hitachi Resource Manager ✔ ✔ ✔ ✔ ✔ - Hitachi Graph-Track ™ ✔ ✔ ✔ - Hitachi FlashAccess ✔ ✔ ✔ ✔ ✔ - Hitachi Virtual Logical[...]

policy-driven automated management across the entire life cycle of shared storage services, ranging from planning, implementation, and pr o visioning, to monitoring. Business Continuity Suite A well-planned business continuity strategy is essential if an organization is to survive planned or unplanned downtime. The Business Continuity Suite include[...]

information and performance levels. The Hi-Track "call home" predictive maintenance system is based on intelligent software that monitors a Lightning 9900 V Series system and notifies the Hitachi Data Systems support center for preventative maintenance. Dynamic Link Manager software solution increases information availability by ensuring [...]

S eries’ systems, Priority Access software and Host Storage Domains guarantee that the most crucial applications get the highest level of service. FlashAccess allows users to quickly and easily lock and unlock information into cache in real time. A user can confidently "fill it up and max it out" to meet performance requirements using t[...]

Hi tac hi Dat a S y s t e m s 38[...]

T h e H i C o m m a n d ™ M a n a g e m e n t F r a m ew o r k a n d H i t a c h i R e s o u r c e M a n a g e r ™ S y s t e m M a n a g e m e n t S o f t w a r e A Pow erfu l an d Open Syste ms Ma nag emen t Phi loso ph y The Hitachi Freedom Storage ™ S oftware Solutions support an enterprise ’ s strategic goal of helping customers focus o[...]

their storage environments than they have ever had in the past. H o wever, most storage managers are faced with administering a myriad of existing components and ne w , complex storage topologies, which lack a clear and consolidated approach to management. It is no longer good enough to pr o vide basic services, such as LUN management, device monit[...]

based on a combination of Hitachi products and best-of-breed storage hardware and software. Likewise, Device Manager software is customizable to integrate easily with existing infrastructures and accommodate future gr owth plans. The Device Manager module simplifies management functions and enables integrated administration of multi-vendor storage [...]

common model with which all vendors can work. A CIM-enabled storage management environment will deliver significant benefits to customers, including the ability to: 1. Overcome the Challenges Created by De-coupling Storage Resources from Specific Server(s) in a SAN N either a single vendor solution nor individual point solutions can meet customers [...]

Over vi ew of Ho w the Sys tem s Man age ment So ft war e Su ppo rts Bus in ess Ob ject ives T o be successful in today ’ s business environment, companies must attain the highest levels of business agility, with maximum IT system efficiency and performance. The “ storage-centric” model of computing is widely accepted today. In this model “[...]

systems, and the user ’ s intranet (public LAN), which allows users to access the Resource Manager software package. 2 . S NM P Su pp or t En s ur es Co mpl i an c e w it h A ll M aj or Sy st em s- m a na ge m en t Pl at fo rm s The Resource Manager software package supports Simple Networ k Management P r otocol (SNMP). SNMP is the most widely us[...]

the operating system as a single, large LUN. This capability simplifies storage management because there are fe wer LUNs to manage. T wo levels of passwor d protection, user and administrator, ensure maximum data security. Users can vie w only Hitachi Freedom Storage LUN configuration information, while administrators can access all LUN configurati[...]

WWN may only be assigned to a single HSD on each Fibre Channel port. SANtinel software may be used to authorize multiple hosts to access a particular HSD. SANtinel software may be further used to permit or deny a particular host to access individual LUNs within the HSD, based upon the WWN of the host. SANtinel software may be used to create up to 1[...]

performed at cache speeds, with no disk latency delay . With FlashAccess software, a portion of cache memory can be allocated to specific data. Users can add, delete, or change FlashAccess software managed data at any time, quickly and easily . D efined by the LUN for open systems, cache data can be as small as a single track or as large as an enti[...]

scrutinizes activity all the way to the logical-device level, reporting on channel- interface-processor usage rates, cache-usage rate by function, and logical-device utilization. This data pinpoints specific activities that may have a negative impact on operations, and lets the user fine-tune the system so that it can be used to maximum advantage. [...]

O ptimizing Performance, A vailability , and Cost with the HiReturn ™ I nvestment Analysis T ool Kit HiReturn ™ Investment Analysis Tool Kit M ost enterprises today require that financial investment analysis be prepared prior to IT capital expenditures. Identifying hardware and software as "best-of-breed" is no longer sufficient justi[...]

Phase 3 – Final report. The HiReturn team provides a business Financial Impact R eport that summarizes the ROI findings of the assessment and outlines the suggested strategy, along with other key recommendations. T he On-site Assessment Produces the Needed Facts to Build a Business Case The HiReturn analysis is conducted at the customer ’ s sit[...]

Global Solutions Services Hi ta ch i D at a Sy ste ms I s C on sis t en t ly R a nk e d Nu mb er On e in th e Ind us tr y In numerous independent sur v eys on IT services organizations, Hitachi Data Systems continually wins the highest ratings in terms of o v erall customer satisfaction. "Service R esponsiveness" is the key Hitachi Data S[...]

G lobal Solutions Services team start by unco v ering the complexity of an individual organization ’ s storage environment in order to define a baseline deployment. Using baseline objectives, the consultants then plan and implement a P A THLINE management environment. In the course of the engagement, product o verview training is pr o vided and t[...]

Hitachi SplitSecond ™ Solutions Rapid Recovery Service for Microsoft ® Exchange The Hitachi SplitSecond ™ Solutions Rapid Recovery Service for Microsoft ® Exchange employs hardware and software in a service package that is designed to solve business problems that can be avoided by the rapid reco v ery for mission-critical applications or data[...]

Infrastructure Planning Service for FICON ™ The Hitachi Data Systems Infrastructure Planning Service for FICON ™ is an on-site, implementation planning and training start up service designed to accelerate the installation of a FICON solution. This service also helps protect current data center infrastructure investments by leveraging the curren[...]

55 Glossary of T erms 10BaseT E thernet with a data transfer rate of 10Mbits/sec. 100BaseT Also known as Fast Ethernet with a data transfer rate of 100Mbits/sec. AG Array Group. ACP Array Control Processor . The group of processors on a PCB that performs the physical access of the disk drives across four FC-AL loops. ACP Pair A combination of two A[...]

A vailability I n computer science, availability refers to the degree to which a system or resource is capable of performing its normal function. A vailability is measured in terms of Mean T ime Between Failure (MTBF) divided by MTBF plus the Mean T ime to Repair (MTTR). The availability equation is expressed as follows: A V AILABILITY = MTBF / (MT[...]

CHA Channel Host Adapter . CHIP Channel Host Interface P rocessor . C–HSN Cache Hierarchical Star Network. CHT Channel adapter T achyon (for Fibre Channel interfaces). Client/Server Architectur e Client/Serv er Architecture is a network architecture in which each computer or processor on the network is either a client or a ser ver. Ser v ers are [...]

CT Q Command-Tag Queuing. DA The Data Adapter resides on the CHT and connects the cache boards to the front end. DAS Direct Attached Storage. Data Availability D ata availability refers to the degree to which a computer system is capable of pro viding data to its users. See also A vailability . Data Copy A term that refers to remote copy, data dupl[...]

DRR Data Recovery and Regeneration. DSS D ecision Support Systems is a term that refers to computer systems used in the areas of business intelligence, such as data warehousing, data mining, OLAP, and others. DT A Data Adapter . ECC Error correction code. ESA/390 ® Enterprise Systems Architecture/390 ® . Often abbreviated S/390 ® , ESA/390 is an[...]

eXtended SCSI Copy (e-Copy) E xtended SCSi is a command in the SCSI 3 specification that allows for the transfer of data without the involvement of the ser ver . The Hitachi Freedom Storage ™ Lightning 9900 ™ V Series implements this command, which is used by programs such as VERITAS ® N etBackup ™ to enable ser verless and LANfree backup . [...]

F reedom Data Networks ™ H itachi Data Systems groups its storage area network (SAN) and network attached storage (NAS) solutions under the label F reedom Data Networks. Freedom Data N etworks pr o vides an open architecture that leverages SAN and NAS technology and offers organizations freedom of choice in deploying data-access and data-sharing [...]

Gigabit Ethernet Pr o vides a standard that supports data transfer at 1000Mbit/sec. Gigabit Ethernet is also called 1000BaseT Category 5 (copper wire) or 1000BaseX (fiber optic). There is a 10,000BaseT version of the Ethernet standard that will be widely available by early 2003. GUI GUI is an acronym that refers to a Graphical User Interface, which[...]

Hitachi Dynamic Link Manager ™ Dynamic Link Manager is a ser v er-based family of software utilities that enhances RAID systems by pr oviding automatic failo v er and load balancing from ser ver-to- RAID channel connection failures. This product allows systems administrators to take advantage of the multiple paths on a Lightning 9900 V Series sys[...]

Hitachi Multiplatform Resource Sharing M ultiplatform Resource Sharing is built right into the Lightning 9900 V Series systems. W ith Multiplatform Resource Sharing, Lightning 9900 V Series systems users can share resources between UNIX ® or Microsoft ® Windows NT ® /Windows ® 2000 serv ers and S/390 mainframe platforms. Sharing resources acros[...]

Hitachi T rueCopy ™ T rueCopy pro vides synchronous or asynchronous remote copy capability for open system and S/390 computers. This allows remote copies o ver virtually unlimited distances. Operating systems that are supported include IBM MVS ® , HP/UX, AIX, S un Solaris, Digital UNIX, Sequent DYNIX/ptx, SGI IRIX, NCR, UNIX SVR4, and Windows NT[...]

InterSan ® P athline ® InterSAN Inc.’ s Storage Area Management (SAM) software product P A THLINE is the first application-based SAM software that aligns the management of storage networks with business policies and objectives. P A THLINE helps customers efficiently manage large, multiprotocol, multivendor storage area networks (SANs) by taking[...]

Logical Unit Number See LUN. Logical V olume The storage medium associated with a logical disk drive. A logical volume typically r esides on one or more storage devices. A host system sees a logical volume as a physical volume, although it does not correlate directly with a physical disk drive. L UN Logical Unit or Logical Unit Numbers. A SCSI term[...]

M etadata D ata about data as used by the CM-HSM and the Control Memory Subsystem of the Lightning 9900 V Series. Mirrored Pair T wo disk units or logical units that contain the same data. The operating system software refers to them as one entity and “ reads from either” and “writes to both” when RAID-1 is enabled. Mirroring A term to desc[...]

NDMP N etwork Data Management Pr otocol (NDMP) is a standard protocol for network- based backup of network-attached storage. NDMP hides the unique hardware interfaces from third-party backup software that allows this software to execute on any NDMP compliant system on the network. Node See Fibre Channel. NVM N on-volatile Memory is a term used to r[...]

Operating system The operating system is the most important software program that runs on a computer . The operating system (OS) performs basic tasks such as recognizing input from a keyboard, sending output to the display screen, keeping track of files and directories on the disk, and controlling peripheral devices, such as disk drive and printers[...]

RAID R edundant Array of Independent Disks. RAID is used to increase the reliability of disk arrays by pr o viding redundancy either through complete duplication of the data (RAID-1, i.e., mirroring) or through construction of parity data for each data stripe in the array (RAID-3, -4, -5). RAID-5, which distributes parity information across all dis[...]

SCSI S mall Computer System Interface. An intelligent bus-level interface that defines a standard I/O bus and a set of high-level I/O commands. There are currently many flavors of SCSI defined by different bus widths and clock speeds. The seven major v ariations of SCSI are SCSI 1, SCSI 2 (Fast/Narr o w), SCSI 2 (Fast/Wide), Ultra SCSI (Fast/Narr o[...]

Sw i t c h A switch is a network device that examines and forwards packets between LAN segments. Synchronous S ynchronous communications occur when the transmission of data between two devices is synchronized with a clocking scheme or other technique. The sender and r eceiver need to synchronize with one another before data is sent. In synchronous [...]

V olume An IBM ® ESA/390 ® term for the information recorded on a single disk unit or r ecording medium. Indirectly, a volume can refer to the unit of recording medium itself. On a non-remov able medium storage device such as a disk drive, the terms may also refer, indirectly, to the storage device that is associated with the volume. When a user [...]

Hitachi Data Systems W orldwide Offices www .hds.com www .storage@hds.com Corporate Headquarters 750 Central Expressway Santa Clara, California 95050-2627 U.S.A. (408) 970-1000 info@hds.com Asia Headquarters Suite 3301-6, Shell T ower T imes Square, 1 Matheson Street Causeway Bay Hong Kong 2525-2385 infoasia@hds.com Australia/New Zealand Headquarte[...]