All Pilot County RFA Implementation Coordinators, SPOCs and Technical Contacts:
The CWS-CARES team has scheduled a hotfix to be deployed to the CWS-CARES production environment. The hotfix deployment will occur on Thursday, January 4, 2024, beginning at 5:00 PM and concluding at 6:30 PM. During the hotfix deployment, the CWS-CARES Resource Family Approval (RFA) Application Submission, Review, and Approval Process will not be accessible for pilot counties Contra Costa, Fresno, Placer, and Santa Clara.
Action Required
Please follow your county process for reporting issues. CWS-CARES County Administrators may contact the CARES Service Desk through the CWDS ServiceNow Customer Support Management (CSM) Portal. Alternate methods are phoning 1-855-292-3444 or via the Service Desk email during normal business hours. You may also email after hours, and a Service Desk representative will contact you the next business day.
CWDS Bulletins
CWS/CMS: Business Objects Server Rebooted
All SPOCs, Technical Contacts, and CAD Contacts:
It was observed that Business Objects (BO) applications were found to be unstable due to BO servers having not been rebooted since December 26, 2023. The cause of the server reboot interruption is under investigation. Multiple Counties contacted the IBM Boulder Help Desk this morning stating that reports were suspended.
The BO team rebooted the BO servers at 8:15 AM this morning. This action has helped restart all BO services again. Counties were able to log in to BO and create reports after the reboot. However, since all counties are now running reports at the same time, it is causing the BO server to run extremely slowly until it catches up with all the pending reports.
BO applications are now stabilizing. Reports should process completely overnight.
The BO team is aware of the BO issues and is working on an upgrade to rectify the issue. The BO team will also reach out to the California Department of Technology (CDT) and inquire for the reason of the server reboot freeze.
Action Required
Review scheduled BO reports today and tomorrow morning. Rerun the report jobs, as necessary.
It was observed that Business Objects (BO) applications were found to be unstable due to BO servers having not been rebooted since December 26, 2023. The cause of the server reboot interruption is under investigation. Multiple Counties contacted the IBM Boulder Help Desk this morning stating that reports were suspended.
The BO team rebooted the BO servers at 8:15 AM this morning. This action has helped restart all BO services again. Counties were able to log in to BO and create reports after the reboot. However, since all counties are now running reports at the same time, it is causing the BO server to run extremely slowly until it catches up with all the pending reports.
BO applications are now stabilizing. Reports should process completely overnight.
The BO team is aware of the BO issues and is working on an upgrade to rectify the issue. The BO team will also reach out to the California Department of Technology (CDT) and inquire for the reason of the server reboot freeze.
Action Required
Review scheduled BO reports today and tomorrow morning. Rerun the report jobs, as necessary.
Microsoft Teams Set Up
All SPOCs and Technical Contacts:
The CWDS Project is excited to announce that Microsoft Teams (MS Teams) will soon be used by specific project teams and workgroups as a collaborative and interactive tool. MS Teams will support document collaboration and interactions between project teams and county personnel included in CARES-related workgroups, as well as core county constituents. The use of MS Teams will be implemented on a phased approach starting with the CARES Analytics Transition (CAT) Workgroup on 1/8/2024, followed by the Data Conversion Workgroup at a later date in January. Updated communications will be sent to provide specific dates as to when subsequent workgroups and service areas will implement the use of MS Teams.
Over the past few months, the CARES Data Infrastructure (CDI) lead engineer, Craig Woody, has worked with 50 counties to complete the technical set up of MS Teams in their county. In order for counties to collaborate with the CWDS Project using MS Teams, both the county and the CWDS Project need to set up a B2B (Business-to-Business) direct connect in their respective Azure Active Directory (AD). It is only Outbound access that the CWDS Project is requesting. The project does not require any Inbound access to a county’s tenant. This process to set up MS Teams can be completed relatively quickly via a meeting with the county’s technical team. During the meeting specific steps can be discussed and completed.
If your county is not actively working with Craig Woody on this initiative to set up MS Teams, we ask that you please do so in order to take advantage of this opportunity for collaboration. Please send Customer Relations an email if your county does not plan to use MS Teams for collaboration with the CWDS Project at this time. Customer Relations will document your county’s decision and/or identify a future timeframe to explore setting up MS Teams in your county.
Action Required
Please reply by 1/19/2024 if your county does not plan to set up MS Teams at this time or in the near future.
The CWDS Project is excited to announce that Microsoft Teams (MS Teams) will soon be used by specific project teams and workgroups as a collaborative and interactive tool. MS Teams will support document collaboration and interactions between project teams and county personnel included in CARES-related workgroups, as well as core county constituents. The use of MS Teams will be implemented on a phased approach starting with the CARES Analytics Transition (CAT) Workgroup on 1/8/2024, followed by the Data Conversion Workgroup at a later date in January. Updated communications will be sent to provide specific dates as to when subsequent workgroups and service areas will implement the use of MS Teams.
Over the past few months, the CARES Data Infrastructure (CDI) lead engineer, Craig Woody, has worked with 50 counties to complete the technical set up of MS Teams in their county. In order for counties to collaborate with the CWDS Project using MS Teams, both the county and the CWDS Project need to set up a B2B (Business-to-Business) direct connect in their respective Azure Active Directory (AD). It is only Outbound access that the CWDS Project is requesting. The project does not require any Inbound access to a county’s tenant. This process to set up MS Teams can be completed relatively quickly via a meeting with the county’s technical team. During the meeting specific steps can be discussed and completed.
If your county is not actively working with Craig Woody on this initiative to set up MS Teams, we ask that you please do so in order to take advantage of this opportunity for collaboration. Please send Customer Relations an email if your county does not plan to use MS Teams for collaboration with the CWDS Project at this time. Customer Relations will document your county’s decision and/or identify a future timeframe to explore setting up MS Teams in your county.
Action Required
Please reply by 1/19/2024 if your county does not plan to set up MS Teams at this time or in the near future.
CWS/CMS: December 2023 Microsoft Patches
All Co-Existent County SPOCs and Technical Contacts:
As requested, here is the list of patches that IBM tested and applied to the Dedicated County Windows 10 Workstations for December 2023:
Windows 10 21H2
MS23-DEC: Cumulative Update for Windows 10 Version 21H2 - KB5033372 (x64)
Windows 10 1607 (LTSB)
MS23-DEC: Cumulative Update for Windows 10 Version 1607 LTSB - KB5033373 (x64)
MS Office 2016
MS23-DEC: Security Update for Microsoft Outlook 2016 - KB5002529
MS23-DEC: Security Update for Microsoft Word 2016 - KB5002520
Browsers
Google Chrome 120.0.6099.110 Available
Action Required
No action is required. Informational only.
As requested, here is the list of patches that IBM tested and applied to the Dedicated County Windows 10 Workstations for December 2023:
Windows 10 21H2
MS23-DEC: Cumulative Update for Windows 10 Version 21H2 - KB5033372 (x64)
Windows 10 1607 (LTSB)
MS23-DEC: Cumulative Update for Windows 10 Version 1607 LTSB - KB5033373 (x64)
MS Office 2016
MS23-DEC: Security Update for Microsoft Outlook 2016 - KB5002529
MS23-DEC: Security Update for Microsoft Word 2016 - KB5002520
Browsers
Google Chrome 120.0.6099.110 Available
Action Required
No action is required. Informational only.
CWS/CMS: California Department of Education - Updated Providers List
All SPOCs and Technical Contacts
On Friday, December 22, 2023, the list of Education Providers in the CWS/CMS application was updated with data obtained from the California Department of Education (CDE). During this update, several Education Providers in the application were end dated. This occurs whenever an existing CWS/CMS Education Provider no longer shows up as an active Education Provider in the data obtained from CDE.
Attached is a list of all closed Public Education Providers and all closed Private Education Providers that were end dated during this update. This information is being provided to assist counties with determining if any data corrections are needed.
Please contact CDE with any questions regarding the attached list:
Email: cdsadmin@cde.ca.gov
Phone: 916-327-4014
Action Required
Review the attached list of closed Education Providers; if the county determines a data correction may be needed, contact the CDE.
On Friday, December 22, 2023, the list of Education Providers in the CWS/CMS application was updated with data obtained from the California Department of Education (CDE). During this update, several Education Providers in the application were end dated. This occurs whenever an existing CWS/CMS Education Provider no longer shows up as an active Education Provider in the data obtained from CDE.
Attached is a list of all closed Public Education Providers and all closed Private Education Providers that were end dated during this update. This information is being provided to assist counties with determining if any data corrections are needed.
Please contact CDE with any questions regarding the attached list:
Email: cdsadmin@cde.ca.gov
Phone: 916-327-4014
Action Required
Review the attached list of closed Education Providers; if the county determines a data correction may be needed, contact the CDE.
New Section on CWDS County Contacts Page – Director Alternate
All Directors and SPOCS:
The CWDS Office is pleased to announce a new section of the County Contacts Page of the CWDS Website. This new section will be implemented on January 29, 2024. After this is implemented, counties will have the ability to add additional contacts for the Director level. This new contact section is called ‘Director-Alternate’. It will be located under the Director tab for both CWS and CARES contacts.
The intent of this new ‘Director-Alternate’ contact is to provide counties the ability to identify contacts that should also receive executive-level communications from the CWDS Project.
On June 20, 2023, a communication was sent to all CWS SPOCs and CARES SPOCs asking for validation of names in the CWS Director area on the CWDS Website County Contact page to ensure that the name listed is indeed your CWS Director. We received responses to this request from fifteen (15) counties with eighteen (18) 'Director-Alternate' contacts. The CWDS Office will add these contacts to the new 'Director-Alternate' section on behalf of those responding counties. If your county did not provide a Director-Alternate contact or needs to make an update, please provide that information on the CWDS Website All Counties page once this section is implemented on Monday, January 29, 2024.
In order to ensure that future communications reach the appropriate audience, we encourage all counties to review and update as needed the County Contacts page on the CWDS Website at All Counties as soon as possible. Attached is the CWDS County Contacts Page Guide that Single Points of Contacts (SPOCs) can utilize as a reference in updating their county’s contacts.
Action Required
After January 29, 2024, add or update contacts to the ‘Director-Alternate’ section as needed. Review and update, as needed, the County Contacts page on the CWDS Website All Counties page. Contact the CWDS Customer Relations team if you have any questions.
The CWDS Office is pleased to announce a new section of the County Contacts Page of the CWDS Website. This new section will be implemented on January 29, 2024. After this is implemented, counties will have the ability to add additional contacts for the Director level. This new contact section is called ‘Director-Alternate’. It will be located under the Director tab for both CWS and CARES contacts.
The intent of this new ‘Director-Alternate’ contact is to provide counties the ability to identify contacts that should also receive executive-level communications from the CWDS Project.
On June 20, 2023, a communication was sent to all CWS SPOCs and CARES SPOCs asking for validation of names in the CWS Director area on the CWDS Website County Contact page to ensure that the name listed is indeed your CWS Director. We received responses to this request from fifteen (15) counties with eighteen (18) 'Director-Alternate' contacts. The CWDS Office will add these contacts to the new 'Director-Alternate' section on behalf of those responding counties. If your county did not provide a Director-Alternate contact or needs to make an update, please provide that information on the CWDS Website All Counties page once this section is implemented on Monday, January 29, 2024.
In order to ensure that future communications reach the appropriate audience, we encourage all counties to review and update as needed the County Contacts page on the CWDS Website at All Counties as soon as possible. Attached is the CWDS County Contacts Page Guide that Single Points of Contacts (SPOCs) can utilize as a reference in updating their county’s contacts.
Action Required
After January 29, 2024, add or update contacts to the ‘Director-Alternate’ section as needed. Review and update, as needed, the County Contacts page on the CWDS Website All Counties page. Contact the CWDS Customer Relations team if you have any questions.
Reports of Outages Impacting CWDS Applications - UPDATE
All SPOCs and Technical Contacts:
December 27, 2023, UPDATE: The connection issue with the CALS feature for CARES-Live has been resolved.
_________________________________________________________________________
December 26, 2023, UPDATE: The California Department of Technology (CDT) has reported that the issue that caused outages for multiple agencies statewide, including the CWS/CMS application and the CARES-Live applications, has been resolved. The CWDS Office has received confirmation that connectivity to the CWS/CMS application and most of the CARES-Live application has resumed. The CWDS team continues to work with CDT to resume connection for the CALS feature of CARES-Live.
_________________________________________________________________________
The CWDS Office has received multiple reports of an outage impacting the CWS/CMS and the CARES-Live applications. The CWDS teams are working with the California Department of Technology (CDT) on a resolution. Additional updates will be sent out as new information is received.
Action Required
CWS/CMS
If users experience any issues with CWS/CMS, please follow your county process for reporting issues or you may contact the IBM Boulder Help Desk at 1-800-428-8268.
CARES-Live
If users experience any issues with CARES-Live, please follow your county process for reporting issues. CARES-Live County Administrators may contact the CARES Service Desk through the CWDS ServiceNow Customer Support Management (CSM) Portal. Alternate methods are phoning 1-855-292-3444 or the Service Desk via email at during normal business hours. You may also email after hours and a Service Desk representative will contact you the next business day.
December 27, 2023, UPDATE: The connection issue with the CALS feature for CARES-Live has been resolved.
_________________________________________________________________________
December 26, 2023, UPDATE: The California Department of Technology (CDT) has reported that the issue that caused outages for multiple agencies statewide, including the CWS/CMS application and the CARES-Live applications, has been resolved. The CWDS Office has received confirmation that connectivity to the CWS/CMS application and most of the CARES-Live application has resumed. The CWDS team continues to work with CDT to resume connection for the CALS feature of CARES-Live.
_________________________________________________________________________
The CWDS Office has received multiple reports of an outage impacting the CWS/CMS and the CARES-Live applications. The CWDS teams are working with the California Department of Technology (CDT) on a resolution. Additional updates will be sent out as new information is received.
Action Required
CWS/CMS
If users experience any issues with CWS/CMS, please follow your county process for reporting issues or you may contact the IBM Boulder Help Desk at 1-800-428-8268.
CARES-Live
If users experience any issues with CARES-Live, please follow your county process for reporting issues. CARES-Live County Administrators may contact the CARES Service Desk through the CWDS ServiceNow Customer Support Management (CSM) Portal. Alternate methods are phoning 1-855-292-3444 or the Service Desk via email at during normal business hours. You may also email after hours and a Service Desk representative will contact you the next business day.
CWS/CMS: IBM Support of Windows 10 SAC v22H2
All SPOCs and Technical Contacts:
IBM has completed its testing of the Windows 10 SAC v22H2 version with the CWS/CMS application. They discovered no issues and are now supporting this version of Windows. The officially supported versions of Windows are now:
Action Required
Counties need to update all of their CWS/CMS workstations to a supported version.
IBM has completed its testing of the Windows 10 SAC v22H2 version with the CWS/CMS application. They discovered no issues and are now supporting this version of Windows. The officially supported versions of Windows are now:
- Windows 10 SAC v22H2 (supported through October 14, 2025)
- Windows 10 SAC v21H2 (supported through June 11, 2024)
Action Required
Counties need to update all of their CWS/CMS workstations to a supported version.
CWS/CMS: Business Objects Server Rebooted
All SPOCs, Technical Contacts, and CAD Contacts:
An issue was discovered this morning with Business Objects (BO). BO applications were found to be unstable due to BO servers having not been rebooted since Thursday, December 21, 2023. Multiple Counties contacted the IBM Boulder Help Desk this morning stating that reports were suspended.
The BO team rebooted the BO servers at 8:30 this morning. Counties were able to log in to BO and create reports after the reboot.
BO applications are now stabilizing. Reports should process completely overnight.
The BO team is aware of the BO issues and is working on an upgrade to rectify the issue.
Action Required
Review scheduled BO reports today and tomorrow morning. Rerun the report jobs, as necessary.
An issue was discovered this morning with Business Objects (BO). BO applications were found to be unstable due to BO servers having not been rebooted since Thursday, December 21, 2023. Multiple Counties contacted the IBM Boulder Help Desk this morning stating that reports were suspended.
The BO team rebooted the BO servers at 8:30 this morning. Counties were able to log in to BO and create reports after the reboot.
BO applications are now stabilizing. Reports should process completely overnight.
The BO team is aware of the BO issues and is working on an upgrade to rectify the issue.
Action Required
Review scheduled BO reports today and tomorrow morning. Rerun the report jobs, as necessary.
CWS/CMS Infected and Unsupported File Extension File Cleanup – UPDATE
CWS Directors, SPOCs and Technical Contacts in Affected Counties:
December 26, 2023 UPDATE: The California Department of Social Services (CDSS), the CWDS Project and IBM have decided that the safest option is to quarantine the identified infected and unsupported files in CWS/CMS. The files will not be deleted from CWS/CMS, but users will be prevented from opening the affected documents. The affected documents will be saved to a quarantined area of CWS/CMS on Saturday, January 6, 2024. If a CWS/CMS user attempts to open one of the quarantined documents, they will receive the following error message:
Please note that this DRMS 00122 error message is informational only and that the user will be able to continue to work in the CWS/CMS application after receiving the error message.
The CWDS Project and IBM are still looking to receive responses from the CWS Directors indicating their permission to permanently delete these files. Please contact CWDS Customer Relations (CWS_CustRel@otsi.ca.gov) by January 31, 2024, with your decision on whether to have IBM delete the files in a batch file deletion process.
__________________________________________________________________________________________
December 18, 2023 UPDATE: In the attached letter the California Department of Social Services (CDSS) has provided updated guidance to Counties impacted by the CWS/CMS infected and unsupported file cleanup effort.
Please contact CWDS Customer Relations (CWS_CustRel@otsi.ca.gov) with any questions related to this guidance.
_________________________________________________________________________________________
As part of planning for CWS-CARES V1, the CARES Data Conversion team has been performing incremental conversion from CWS/CMS data and documents into CWS-CARES. After the documents are extracted from CWS/CMS, they are scanned to ensure that no viruses are introduced into the CARES environment. During this process, it was discovered that, out of 73 million documents that are stored in the CWS/CMS database, approximately 2,400 documents are infected with a virus or malware. Of these 2,400 documents, 95% were last updated in 2001 or before.
Further, there are an additional 2,600 files that have an unsupported file extension.
Due to security concerns, these documents and files cannot be moved into the CWS-CARES application.
Please NOTE: These infected documents are stored in binary large object (BLOB) format on the mainframe in the Db2 database. As a result, these documents do not affect CWS/CMS data on the mainframe. However, if the document is opened, the virus can infect the user’s personal computer. Please ensure that you have a current antivirus operating on your workstation.
If you are receiving this communication, it means that your county is impacted by this issue.
For each affected County, we will send a report listing the infected files and unsupported file extension files, with additional details pertaining to that County. The reports will be sent to affected counties after the Kick-off meetings.
Please provide a point of contact including IT Support and Security Officer as appropriate and their information (Name, Email, Role, etc.) for the person that should receive your County’s report no later than 11/8/23. The report will be sent via encrypted email to that person.
Please use this form to provide the point(s) of contact: CWS/CMS Infected Documents/Unsupported File Extension - Contacts
Once Customer Relations receives the point of contact person’s information, they will be invited to the Kick-off meetings scheduled for 11/14/23 and 11/16/23. The intent of those meetings will be to review the next steps.
Action Required
Please provide a point of contact and their information (Name, Email, etc.), using the link above, for the person that should receive your County’s report no later than 11/8/23.
December 26, 2023 UPDATE: The California Department of Social Services (CDSS), the CWDS Project and IBM have decided that the safest option is to quarantine the identified infected and unsupported files in CWS/CMS. The files will not be deleted from CWS/CMS, but users will be prevented from opening the affected documents. The affected documents will be saved to a quarantined area of CWS/CMS on Saturday, January 6, 2024. If a CWS/CMS user attempts to open one of the quarantined documents, they will receive the following error message:
Please note that this DRMS 00122 error message is informational only and that the user will be able to continue to work in the CWS/CMS application after receiving the error message.
The CWDS Project and IBM are still looking to receive responses from the CWS Directors indicating their permission to permanently delete these files. Please contact CWDS Customer Relations (CWS_CustRel@otsi.ca.gov) by January 31, 2024, with your decision on whether to have IBM delete the files in a batch file deletion process.
__________________________________________________________________________________________
December 18, 2023 UPDATE: In the attached letter the California Department of Social Services (CDSS) has provided updated guidance to Counties impacted by the CWS/CMS infected and unsupported file cleanup effort.
Please contact CWDS Customer Relations (CWS_CustRel@otsi.ca.gov) with any questions related to this guidance.
_________________________________________________________________________________________
As part of planning for CWS-CARES V1, the CARES Data Conversion team has been performing incremental conversion from CWS/CMS data and documents into CWS-CARES. After the documents are extracted from CWS/CMS, they are scanned to ensure that no viruses are introduced into the CARES environment. During this process, it was discovered that, out of 73 million documents that are stored in the CWS/CMS database, approximately 2,400 documents are infected with a virus or malware. Of these 2,400 documents, 95% were last updated in 2001 or before.
Further, there are an additional 2,600 files that have an unsupported file extension.
Due to security concerns, these documents and files cannot be moved into the CWS-CARES application.
Please NOTE: These infected documents are stored in binary large object (BLOB) format on the mainframe in the Db2 database. As a result, these documents do not affect CWS/CMS data on the mainframe. However, if the document is opened, the virus can infect the user’s personal computer. Please ensure that you have a current antivirus operating on your workstation.
If you are receiving this communication, it means that your county is impacted by this issue.
For each affected County, we will send a report listing the infected files and unsupported file extension files, with additional details pertaining to that County. The reports will be sent to affected counties after the Kick-off meetings.
Please provide a point of contact including IT Support and Security Officer as appropriate and their information (Name, Email, Role, etc.) for the person that should receive your County’s report no later than 11/8/23. The report will be sent via encrypted email to that person.
Please use this form to provide the point(s) of contact: CWS/CMS Infected Documents/Unsupported File Extension - Contacts
Once Customer Relations receives the point of contact person’s information, they will be invited to the Kick-off meetings scheduled for 11/14/23 and 11/16/23. The intent of those meetings will be to review the next steps.
Action Required
Please provide a point of contact and their information (Name, Email, etc.), using the link above, for the person that should receive your County’s report no later than 11/8/23.
If unable to find a specific Bulletin, please contact Communications.