Fixed issues

The following table lists the issues that were fixed in PowerFlex 3.5.1.

NOTE: If an issue was reported by customers, the customers' Service Request numbers appear in the "Issue number & SR number" column, and serve to correlate between customer-reported issues and the PowerFlex issue number.
Table 1. Fixed issues
Issue number & SR number Problem summary
SCI-55850 When performing log collection using tool, if the file system name is too long a df command in the log collection will fail, which will fail the log collection for this node.
SCI-55828 When performing VMware-based storage migration of a VM running on a VASA-based datasore (volume) to another such datastore, the operation fails.
SCI-55691 Non-disruptive upgrades (NDUs) from versions 3.0-3.0.1 to higher versions might cause a Data Unavailable event. This applies to systems that use Fine Granularity Storage Pools with SDS device paths like: /dev/disk/by-id/<device name> or /dev/disk/by-path/<device name> or /dev/mapper/<device name>. Some of the devices encrypted with Cloudlink might fail.
SCI-55657 During SDR restart and failover to another SDR, there might be a Data Integrity issue in the replicated copy.
SCI-55656 When setting up a system with replication, if the admin creates an RCG (Replication Consistency Group) prior to adding an SDR, and the master MDM is restarted, the initial copy process will get stuck.
SCI-55596 The presentation server's default certificate expires 90 days after the presentation server was installed.
SCI-55545 PowerFlex API doesn't expose the properties for the RPL_CAPACITY_ALERT_LEVEL query.
SCI-55527, SCI-55528 When deploying or upgrading a system of 100 nodes or more, the PowerFlex Gateway refresh might take several minutes or the upgrade might fail due to long query time. This is caused by password encryption for JSON serialization, which slows down the process.
SCI-55505 It is not possible to install the presentation server without sudo installed on the server.
SCI-55466 Various Apache Tomcat security vulnerabilities addressed in 8.5.57 release.
SCI-55450 After upgrading a Windows-based PowerFlex Gateway, the lockbox is locked, and content cannot be read from it.
SCI-55448 In rare cases, during the NDU workflow, the PowerFlex Gateway reports that the SDS failed to exit maintenance mode.
SCI-55361 In a rare scenario where replication is running and an SDR is entering or exiting Maintenance mode, while both the SDR and the MDM are restarted, the initial copy might get stuck.

SR# 18115954

During NDU to PowerFlex v3.5, there is a large amount of 256 KB read I/Os against the SDS backend disk devices driven by a temporary background metadata update.

When upgrading larger systems (over 200 nodes) with AutoRefresh selected, the PowerFlex gateway might experience some of the following issues:

  • High CPU utilization
  • Slow NDU status update in the UI
  • Prematurely presenting a phase complete
SCI-55184 When trying to stop the presentation service for maintenance operations, it might not stop successfully. The service might seem to be stopped, even though the process is still running when using ps -ef.
SCI-55150 When performing VM cloning on a datastore running on vVol mapped volumes, some VMs might not start successfully after the clone operation is finished.

SR# 17991736

When using an IPv6 network between the PowerFlex Gateway and the MDM, during network failure, the PowerFlex Gateway fails to reconnect, resulting in an HTTP 500 error via REST API.
SCI-55062 In rare cases, during MDM switchover, if additional SDSs fail, one or more SDSs might be decoupled and will not be able to reconnect to the system.
SCI-55020 Upgrade from PowerFlex 3.0.x might cause the PowerFlex Gateway to fail with an error message: "unexpected error has occurred".
SCI-55016 The command scli --query_system_limits has output that differs from documentation:

The maximum number of SDRs as documented is 64, and not 512 as listed in the command output.

SCI-54914 In rare cases, an SDR might fail, and show the following error in the exception file:

Panic in file /data/builds/workspace/ScaleIO-Common-Job/src/mos/umt/mos_umt_uthread.c, line 1370, function mosUmt_ResumeUThread, PID 80440.Panic Expression pUmt->state == MOS_UMT_STATE__SUSPENDED

SCI-54913 The presentation server has excessive logging of the SDS device data. The size is 8 KB data per device, per minute.

SR# 17933245

When removing volumes and running the REST call: /api/instances with a GET request, the request returns an http 500 status with an invalid JSON object.
SCI-54749 In rare cases, during RCG removal, the Failover test volume will not be automatically removed from the system.
SCI-54601 During a PowerFlex Gateway upgrade, an error might appear if a firewall is not installed.

Error example: finished with error: firewall-cmd not found.* error: %preun(EMC-ScaleIO-gateway-x.x-nnn.nnn.x86_64) scriptlet failed, exit status 255

SCI-54588 In HCI systems with Fine Granularity Storage Pools, the SDS might fail due to insufficient RAM. This issue occurs only in HCI Fine Granularity-based systems. It will occur during the SDS upgrade flow as part of NDU, or after a fresh deployment when restarting the SDS.
SCI-54561 When running the command: scli --query_system_limits it returns wrong output for Number of fault sets:

Max: 32768 (wrong value)

The correct value is:

Number of fault sets per protection domains: Max: 64

SCI-54557 When running on AIX, the file get_info_run.log is missing.

SR# 17734455

Attempting to initiate log collection using the PowerFlex Gateway fails with the error "Server didn't respond, code 400".
SCI-54519 I/O errors during V-Tree migration if both source and destination Storage Pools have inflight checksum disabled, and a user enables inflight checksum on the destination Storage pool while migration is in process.

SR# 17823669

The following event is received in the MDM: SDS_DEV_MOVED_TO_FAILURE_STATE. The events continue to be reported even after the device has entered Error state.
SCI-54412 During NDU of large scale systems, the PowerFlex Installer might fail exit maintenance mode for SDS, stating at least one SDS device is in error state.
SCI-54385, SCI-53114, SCI-54880 In AIX-based systems, when there is a high I/O load, the SDC might fail.
SCI-54363 In rare cases, during SDR enter\exit maintenance mode, the SDR might fail with the following exception: Panic in file /data/builds/workspace/ScaleIO-Common-Job@2/src/sdr/obj_mngr/obj_mngr.c, line 365, function baseObjMngr_ObjRemovalTimerExpired
SCI-54358 Application I/O is prioritized over Initial Copy I/O. This minimizes the effect that Initial Copy I/O load has on the RPO adherence of other RCGs and pairs, but on the other hand, may slow Initial Copy progress when there is a high I/O load.
SCI-54233 Online help link in WebUI points to a blank page.
SCI-54226 SDR might fail with the following error in the exception file:

Panic in file /data/builds/workspace/ScaleIO-Common-Job/src/sdr/replicator/replicator_cg.c, line 1572, function replicatorCg_UpdateClose

SCI-54212 When running /opt/emc/scaleio/sdc/bin/drv_cfg --mod_mdm_ip --ip <some_non_existing_mdm_ip_address> --new_mdm_ip --file /etc/emc/scaleio/drv_cfg.txt

instead of modifying an IP address, a new line is added as if it were a new system.

SCI-54105 During creation of a replication initial copy, if the MDM is unavailable, replication might freeze for the relevant pair.
SCI-54073 Running log collection via the PowerFlex Gateway to a Windows SDC might fail.
SCI-54025 When enabling Restricted SDS Mode followed by SDC installation, SDC is not shown in the web interface or SDC might appear as disconnected. When trying to approve it, an error message is returned stating that it is already approved.
SCI-53898 In some cases, the Web GUI is not able to login to the system after several days due to a Chrome extension issue.

SR# 17456691

When using the "Native and LDAP" authentication method, most of the CLI events by the user admin are logged by MDM with "User: ''

0x7fc5a6645db8:mosEventLog_PostInternal:00608: New event added. Message: "Command query_users received, User: ''. [36020990]". Additional info: "" Severity: Info


SR# 17581338

When adding the SDS host's system disk as an SDS device, if the device file is /dev/sda, the device cannot be added, with the error message:

Error: MDM failed command.

User Content is present on device


In rare cases, during RCG removal, the SDR might fail with the following exception: Panic in file /data/builds/workspace/ScaleIO-Common-Job/src/sdr/obj_mngr/obj_mngr.c, line 365, function baseObjMngr_ObjRemovalTimerExpired

The SDR watchdog recovers the process.

SCI-53722 In rare cases, on a system where the replication feature is active, if an SDR is disconnected due to the firewall blocking the port to remote SDR, the MDM might fail at the source site, with the following exception: Panic in file /data/builds/workspace/ScaleIO-Common-Job/src/mdm/replication/peer_mdm/mdm_mdm_msg.c, line 288, function mdmMdmMsg_TimerExpired
SCI-53716 The background scanner does not correct the corrupted the data as expected. The following appears in the MDM event log:

SCANNER_READ_REPORT ERROR Background device scanner reported read error on SDS: SDS3, Device: /dev/sdf (Found: 62, Fixed: 0)

SCI-53650 While replication is running when multiple failures occur both on nodes and on the network, SDR might fail with the exception:

data/builds/workspace/ScaleIO-Common-Job/src/sdr/replicator/replicator_reporter.c, line 1907, function replicatorReporter_UpdateCe

SCI-53344 While replication is running, if an SDR fails or is disconnected, an I/O spike is seen every 120 seconds and can have a short impact on replication.
SCI-53022 Clearer descriptions of LDAP settings in PowerFlex Gateway.

SR# 16651864

Login to the PowerFlex Gateway is not possible using LDAP when the LDAP search is configured to use the option "--base_dn_list". The login operation to the PowerFlex Gateway returns the message:

Error: Unable to authenticate with LDAP: The URL does not match the corresponding setting in the Lockbox

SCI-51985 In rare cases, the SDR might fail with exception: Panic in file /data/builds/workspace/ScaleIO-Common-Job/src/sdr/obj_mngr/obj_mngr.c, line 365, function baseObjMngr_ObjRemovalTimerExpired

SR# 15616808

After running the "Run script on Host" capability (also known as operating system patching) from the PowerFlex Gateway, the following SNMP message will be constantly sent:

INFO c.e.s.s.s.snmp.TrapSender - SNMP Alert for entity = [0], alert = [System.Credentials.MDM_CREDENTIALS_ARE_NOT_CONFIGURED] code = [SIO01.04.0000002] sent successfully


SR# 15547075 v754914

ESXi might hang in a highly intensive I/O load when a single VM is the only active virtual disk on a datastore\LUN. The SDC-exposed controller's I/O request queue is filled. This causes a kernel slab resource to become exhausted and, consequently, causes requests to fail.

SR# 16718335

During upgrade from v2.5.1 to v3.0.201 on a fully utilized cluster, the PowerFlex Gateway fails to place an SDS in IMM due to insufficient resources.

SR# 14510727

In extremely rare cases, the MDM might fail due to the OS disk being too slow. In such a case, the MDM process might still keep the sockets open to the SDCs, even though a switchover occurred. Such a failure prolongs the MDM disconnection from SDCs, and might cause I/O errors.
SCI-44997 Addition of pre-partitioned NVMe disks to a PowerFlex system will cause removal of the partitions instead of failing the operation.